Note to Users

NOTE TO USERS

The original manuscript received by UMI contains pages with indistinct print. Pages were microfilmed as received.

This reproduction is the best copy available

Nationai Library BiMmthèque nationale du Canada Acquisitions and Acquisitions et Bibliographic Services senrices bibliographiques 395 weningm Street 395, me Wellington Ottawa ON K1A CHU4 atawaûN K1AON4 canacta Canada

The author has granted a non- L'auteur a accordé une licence non exclusive licence allowing the exclusive permettant à la National Lhrary of Canada to Bibliothèque nationale du Canada de reproduce, loan, distn'bute or sell reproduire, prêter, distri'buer ou copies of this thesis m microform, vendre des copies de cette thèse sous paper or electronic foraiats. la forme de microfiche/nlm, de reproduction sur papier ou sur format électronique.

The author retains ownership of the L'auteur conserve la propriété du copyright in this thesis. Neither the droit d'auteur qui protège cette thèse. thesis nor substantial extracts fiom it Ni la thèse ni des extraits substantiels may be printed or otherwise de celle-ci ne doivent être imprimés reproduced without the author's ou autrement reproduits sans son permission. autorisation. Contents

List of Tables ...... v

List of Figures ...... vi

List of Abbreviations and Symbols ...... ix

Acknowledgments ...... xi

Abstract ...... xii

1 Introduction ...... 1 1.1 SPECTImaging ...... 1 1.2 MR Imaging ...... 2

2 Background ...... 5 2.1 The Registration Problem ...... 5 2.1.1 Imaging Geometry ...... 5 2.1.3 What is Registration ...... 7 2.2 TheAbsoluteOrientationProblem ...... 10 2.3 Descriptions of Rigid Rotation ...... 15

3 Theory and Implementation of the New Method ...... 18 3.1 Non-iterative Direct Method ...... 18 3.2 Revisiting Methods Based on the SVD ...... 23 3.3 Reference System ...... 26 3.4 Preliminary Studies ...... 27 3.5 Reference System Orientation ...... 28 3.5.1 Image Segmentation ...... 30 3.5.2 Valid versus Invalid Images ...... 32 ... 111 3.6 Determination of Scale ...... 33 3.6.1 Volume Rendering ...... 34 3.7 Determination of Translation and Rotation ...... 35 3.7.1 Fittingastraight Linetoa3-D9C1oud" ...... 35 3.7.2 Translational Part ...... 39 3.7.3 Rotational Part ...... 39 3.8 Applying the Registration Transformation ...... 43 3.8.1 Mu1tilinea.r Interpolation ...... 15 3.9 Implementation ...... 50

4 Experirnental Results ...... 52 4.1 Experimental Setups and Data Sets ...... 52 4.2 Strategy for Accuracy Estimation ...... 53 4.3 SPECT-teSPECT Registration ...... 54 4.4 SPECT-bMRRegistration ...... 57 4.5 MR-to-MR Registration ...... 55

5 Conclusions ...... 61 1 Discussion ...... 61 5.1.1 Reference System Design ...... 62 5.1.2 Interpretation of Error Measurements ...... 62 5.2 Possible Applications ...... 63 5.3 Future Development ...... 64

References ...... 65

Appendix A Quaternion Mathematics ...... 68 A.l Conjugate and Norm of Quaternions ...... 69 A.2 Product of Quaternions ...... 69 A.3 Division of Quaternions ...... 70

Appendix B Program Listings ...... 71 B.1 First ModuIe ...... 71 B.2 Second Module ...... 76 List of Tables

3.1 Example of coordinates (in pixels) of reference points for an MRI data set ...... 37

4.1 Example of a registration enor table for an MRI data set ...... 55 4.2 Angles of rotation for the SPECT-teSPECT registration ...... 56 1.3 Angles of rotation for the SPECT-teMR registration ...... 58 4.4 Statistics on the MR-teMR registration ...... 59 4.5 Angles of rotation for the MR-t*MR registration ...... 60

A .1 Quaternion multiplication table ...... 69 List of Figures

1.1 SPECT imagingscheme ...... 1.2 A transaxid SPECT Image...... 1.3 A transaxial MR Image ......

2.1 Imaging geometry...... 2.2 Imaging orientations . S-sagittal, C.coronal, T.transaxia.l ...... 2.3 SPECT Imaging . (a) Placement of the imaging planes . (b.c) Two adjacent slices ...... 2.4 MR Imaging . (a)Placement of the imaging planes . (b.c) Two adjacent slices...... 2.5 Orientations and spacings of the section planes in different machines . 2.6 Steps for registration ...... 2.7 Corresponding MRI and SPECT slices...... 2.8 Problem: match triangles ...... 2.9 Reference points. known positions ...... 3-10 Reference points. unknown positions ...... 2.11 Tree diagram illustrating the classification of the computat iond met h- ods for solving the absolute orientation problem ......

3.1 Structure of triangles ...... 3.2 Unique position of a rigid body ...... 3.3 Left to right: initial position, rotated position. flipped position .... 3.4 Solid angle of plausible rotations ...... 3.5 Acquired data...... 3.6 Difficulty in using of vertices as reference...... 3.7 Reference system prototype ...... 3.8 Problems encountered in preliminary studies. (a) Marginal slice is "empty* . (b) Vertex lies between slices. (c) Vertex does not belong to the set . (d) Tubing curvature imposes a problem ...... 3.9 Marginal markers added to the reference system ...... 3.10 Location of reference points in images is a function of proximity to the triangle vertex ...... 3.11 (a) Original monochrome image . (b) Blurred image thresholded at 10% of the maximum intensity. (c) Original image thresholded at 10% of the maximum intensity...... 3.12 (a) Mask image. (b) Result of masking . (c) Masked image thresholded at 30% of the maximum intensity ...... 3.13 Close-up of the tubing crossed by a slice ...... 3.14 Reduction of the 3-D line-fitting problem to 2-D cases in planes YZ andXZ...... 3.15 Vectors representing sides...... 3-16 Rotation in terrns of quaternions ...... 3.17 Finding the rotational part of the registration transformation as a combination of two rotations . (a) First rotation . (b) Second rotation ... 3.18 Voxel analysis in an MR image ...... 3.19 Voxel analysis in a SPECT image ...... 3.20 Pixelation effect and "edgy" transitions ...... 3.21 Explanation for the pixelation effect and "edgy" transitions ...... 3.22 One-dimensional cross.section ...... 3.23 Won-interpolated data...... 3.24 Goal of interpolation ...... 3.25 Neighbouring supervoxels ...... 3.26 Influence of neighbours ...... 3.27 Computation of the grey scale value of a new voxel . (a) In a "level" .(b) Between the " levels" ...... 3.25 Results of interpolation . (a)SPECT image, no interpolation . (b)SPECT image. interpolated ......

4.1 SPECT-to-SPECT registration . (a) Benchmark image. (b) Registered image . (c) Result of subtraction ......

vii 4.2 SPECT-to-SPECT registration error surfaces. (a) " Idle triangle". (b) " Reference triangle"...... 57 4.3 SPECT-twMR registration. (a) Benchmark image. (b) Registered image. (c) Result of subtraction...... 57 4.4 SPECT-to-MR registration error surfaces. (a) "Idle triangle". (b) " Reference trianglen...... 58 4.5 MR-t-MR registration. (a) Benchmark image. (b) Registered image. (c) Result of subtraction...... 59 1.6 MR-to-MR registration error surfaces. (a) " Idle triangle". ( b) " Refer- ence trianglen...... 60 List of Abbreviations and Symbols

MRI Magnetic Resonance Imaging ...... 1 SPECT Single Photon Emission Cornputed Tomography ...... 1 PET Positron Emission Tomography ...... 4 PSF Point Spread Function ...... 1 SNR Signal to Noise Ratio ...... 2 Radio Frequency ...... 3 SVD Singular Value Decomposi t ion ...... 16 DVR Direct Volume Rendering ...... 31 Coordinate system of the scanner ...... 9 Coordinate system of the body (head)...... 9 Sagittal, Coronal . and Transaxial irnaging orientations ...... 5 ICP Iterative Closest Point Algorithm ...... 12 Rotation matrix ...... 16 Cross covariance rnatrix of point sets {n}and {fi)...... 16 Translation ...... 30 Scaling ...... 90 first moments of the {Zi} and {fi)...... 16 Variances around the mean vectors Cr and Gu, ...... Y4 Mean squared error of two point patterns ...... 23 Resolution of the MRI set along the .Y avis ...... 14 ymres Resolution of the MRI set along the Y axis ...... 44 Resolution of the MRI set dong the Z axis ...... 44 xsres Resolution of the SPECT set dong the .Y axis ...... 44 ysres Resolution of the SPECT set dong the Y axis ...... 44 Resolution of the SPECT set along the Z axis ...... 44

Acknowledgments

1 would like to thank Dr. P.H. Gregson for his supervision during this project. Dr. G. Mawko and Dr. M. Butler, thank you for the help with the images. 1 want to thank rny wife and friend Yulia for love, help and faith and al1 the students of the CVlP Lab for their participation and fruitful discussions. 1 would aiso like to express gratitude to the National Science and Engineering Research Council for partial financial support during t his project . Abstract

There are many tedious tasks in medical imaging t hat can and should be automated. Often there is a need to observe and measure changes in structure or functioning of an interna1 organ. To perform this task images taken at different moments in time must be acquired with the same orientation of the imaging system with respect to the patient's body. The task of combining functional adstructural data in one image is even more dernanding because often images of different modalities dso have different resolutions. This thesis proposes a new method for registration of brain images that meets the described requirements. The developed algorithm performs mathematical rotation and re-sampling of one 3-D data set to match the orientation and the sampling rate of the other data set. A new, light-weight adjustable reference system was designed to fit inside various scanners. Once adjusted, it can be mounted on a human head, taken off and then remounted in the some position. A new algorithm was developed to determine the position and orientation of the reference system inside the imaging system. The algorithm segments each image in each of two data sets, treating marks left by the reference system as the useful signal and the rest of the information as noise. The novelty of the suggested approach is that it uses the redundancy of the information in the data set. The vector-quaternion form in which rotation is found requires as few as four readings from the reference system to determine relative position and orientation. This permits designing the segmentation process to minimize noise. Redundant information is used to increase the accuracy of determining the position of the reference system. A modification of the direct volume rendering technique is then used to obtain images at the proper scale. Multi-linear interpolation is applied to diminish spatial quant kat ion effects caused by different spatial resolutions of the data sets and by non-cubic voxels. A number of applications of the new method include intra-patient sarne modality registration and intra-patient different modality registration.

xii Chapter 1

Introduction

This chapter describes two imaging modalities used to provide data for this research. narnely, SPECT (Single Photon Emission Computed Tomography), and MRI (Mag- net ic Resonance Imaging). Physical principles of t hese two imaging techniques are presented dong with the basic notions and important characteristics of the equip ment. One of the obvious and desired goals of medical imaging in general is to acquire the most cornprehensive picture of an interna1 structure possible. The picture should include bot h structural and functional information. This would provide doctors wi t h a powerful diagnostic instrument as well as improved information for planning and evaluating therapy, so t hey could t hen determine the value of a particular treatment by simply comparing images of the organ taken at different moments during the course of treatment, rat her t han by deducing its effect iveness t hrough the analysis of symptoms. This t hesis presents a solution to the regist ration problem which permi ts combining different types of information into one image.

1.1 SPECT Imaging

The goal of single photon emission, computed tomography (SPECT) is to reconstmct the three-dimensional distribution of radioactivi ty inside the patient's body from projections. Different projections are acquired by counting the number of photons collected on a planar, collimated detector placed at various positions around the body (Fig. 1.1). These projections are not simple Line integrals but have point- spread functions (PSFs) due to the non-perfect coiiimation of the detector and to the attenuation of the photon's energy during its flight from source to collimator. In addit ion, Compton scattering, gamma-ray attenuat ion and ot her physical phenomena make the data noisy thereby lowering SNR (Signal to Noise Ratio). Various prereconstruction and post-reconstruction techniques have been used to enhance S P ECT data.

Figure 1.1: SPECT imaging scheme.

The t echniques employ met hods such as constrained deconvolut ion, Wiener filtering and count-dependent Metz filtering [6]. Maximum entropy and hornomorphic filters have aiso proved to produce good results [IO]. There is a number of important assumptions associated with the use of data coming from SPECT imaging machines. As a rule, structural interpretation becomes difficult due to the very nature of the method. This happens because the general aim in nuclear medicine is to study physiological activity of the imaged organ and not its structure. SPECT images are often noisy. Usually the best resolution one can expect from SPECT images ranges from 4 to 10 mm/pixel. SPECT data, therefore, can be characterized as noisy, having poor resolution and showing function rather than anatomy. An example of an image acquired using a SPECT scanner is shown in Fig. 1.2.

1.2 MR Imaging

Magnetic resonance imaging (MRI) is based on a different physical principle. This branch of medical irnaging is growing md ha becornc icry popiilar because this Figure 1.2: A transaxid SPECT Image. method is completely non-invasive. An MRI machine measures and images the density of isolated protons in the tissues of the body. This is done by measuring the intensity of the magnetic field emitted by the tissue after an excit ing magnetic field is removed. This technique is based on the fundamental property that protons possess a magnetic moment and spin. When placed in a magnetic field, the proton precesses about this field. Initially al1 the protons are aligned either parallel or antiparallel to the magnetic field. When a radio frequency (RF) signal having an appropriate strength and frequency is applied to the object, the protons absorb energy, and more of them precess to the antiparallel state. When the applied RF signal is removed, the absorbed energy is reemitted and is detected by an RF receiver. The proton density in the subject can be determined from the characteristics of the received signal. By controlling the applied RF signal and the surrounding magnetic field, these events can be made to provide information on local proton densities throughout the body. Magnetic resonance is now widely used in medicine to observe inner structures in the organs of the human body. Typical resolution of MRI machines is close to 0.5 mmlpixel. An example of an image acquired using MRI imaging is showo in Fig. 1.3. There are three ways to accomplish the major goal of collecting the most comprehensive picture of an intemal organ. The first is to drastically improve the resolution of SPECT irnaging. This requires the creation of new machines with better sensjtivity which will make possible more precise determination of the structure of the examined organ. Figure 1.3: A transaxial MR Image.

The second path is to implement functional MRI. As described in [9], it is possible to determine concentration of certain substances such as lactate (products of neurophysiologicai activity) directly from the MRI scan. It has been determined pl] that during visual and auditory stimulation, the local oxygen consumption increased by merely 5% whereas the local glucose metabolic rate in the hurnan cortex increased by about 50% and the local blood flow also increased by 50%. This suggests anaerobic glycolysis in stimulated regions. There is no adequate explanation for this result. Since lactate is the end-product of anaerobic metabolism, it might be used as an indi- cator of stimulated neural activity similar to PET (Positron Emission Tomography) but without the radiation associated with PET. Detection of lactate (or changes of lactate) within specific brain regions in response to stimulation should make it possible to use spect roscopic imaging to investigate stimulated brain funct ion. This class of methods and the specidized equipment required will be developed in future. Findly, there is the third way of getting desired functional and anatornicai data with the current state of imaging technology. This method uses superimposition or "fusion" of the data sets from MR imaging and SPECT imaging. Wi th funct ional information added to the corresponding structural information, active regions of the particular structure cari be determined. This leads to the requirement for intermodal image registration, the solution of which is a substantial part of this thesis. Chapter 2

Background

SPECT images provide functional data while MR imaging reveals structure. To obtain a more thorough picture of an internd organ (in our case the human brain), the two sets of data should be superirnposed. This cannot be done directly. Necessary premises for the solution of the registration problem require in-depth knowledge of the two types of data set that have been used in this research. In this chapter a description of how the digit ization is performed in MR and SPECT scanners is given. A short survey of various techniques used for registration of data sets is provided and a classification of various methods is presented.

2.1 The Registration Problem

2.1.1 Imaging Geometry

Both sets of data are composed of two-dimensional images that are the projections on the XY plane of slices of three-dimensional volumes. Planes containing slices are al1 parailel to the XY plane. Slice thichess is determined by the technical capabilities and adjustments of the SPECT or MR equipment that is used to obtain the data. The imaging geometry is shown in Fig. 2.1. For example, a cylinder with its axis parallel to the Z axis would produce a series of identical images al1 showing circles of the same diameter. MR equipment allows arbitrary orientation of the magnetic field and, therefore, imaging plane. SPECT machines ars less flexible and only three main orientations can br ottained. These are tramaxial (T). coronal (C),and sagittal (S)orientations which are shown in Fig. 2.2. BodyY-\ of a patient

Figure 2.1: Imaging geometry.

A single slice can be visualized as that volume of the body lying between two parallel planes. SPECT and MR machines treat this thin "slabn as a twedimensional "slice" . Both machines quantize the slice into pixels. Thus, each element of a slab is a voxel (element of volume with X and Y dimensions given by the spatial quantization of the slice, and the Z dimension given by the slice thickness. In t his thesis X and Y directions indicate the directions in the plane of a slice, unless it is stated otherwise. There are differences between SPECT ond MR machines with respect to the distance between slices. In SPECT imaging, sequential slices of the digit ized volume are contiguous. Al1 of the physical volume in the field of view of the scanner is digitized. Resolution in the X, Y, and Z directions is 10 mrn/pixel. Thus, voxels for the SPECT method are cubic with dimensions 10x10~10mm. Fig. 2.3 shows two adjacent tramaxial slices acquired using SPECT equipment. On the other hand, MR imaging allows each slice to be much thinner than for SPECT. Slice thickness (resolution in the Z direction) cm be comparable to the resolutions in X and Y directions. In the studies performed in this research, resolution in the XY plane ranged from 0.7 to 1.0 mm. It is theoretically possible to require the resolution in the Z dimension be equal to the resolution in the XY plane. This wil1 result in cubic voxels. The negative effect is that voxels of srna11 volume mean low SNR. Therefore, there is a tradeoff between voxel volume and SNR. In clinical practice MR imaging systems are usually configured for voxels to span 0.5 - 0.9 mm Figure 2.2: Irnaging orientations. S-sagittal, C-coronal, T-transaxial. in the X and Y directions, and 4 - 7 mm in the Z direction. This is because higher resolution in the Z direction requires much greater imaging times which poses clinical difficulties. Thus, an image of the head consists of 20-30 slices. Clinicians sacrifice sorne accuracy, because of tbick slices. but it seems that this resolution is sufficient for current clinical practice. Fig. 2.4 shows two adjacent trmsaxial slices that were acquired using the MRI machine. Though technically it is possible to tune the MRI machine in such way as to produce cubic voxels, it was decided to use the same imaging parameters used in current clinical practice. This is done to reduce the volume of data processed while covering the same physical volume, to avoid retuning the scanner, and also to ease evaluation of the results. The standard tuning of the MR scanner produces voxels that are rectangular prisms with dimensions 0.7 x 0.7 x 6.6 mm dong the X,Y, and Z axes respectively.

2.1.2 What is Registration

Registration of the image sets prior to superimposing them is necessôry because generdly the imaging planes of SPECT and MRI machines are not parallei. This is because the patient's head is positioned differently in the two machines. Images also Figure 2.3: SPECT Imaging. (a) Placement of the imaging planes. (b,c) Two adjacent slices.

Figure 2.4: MR Imaging. (a) Placement of the imaging planes. (b,c) Two adjacent slices. have different resolutions. An example is given in Fig. 2.5. The pardlel lines rep resent slices. To visualize the problem better, the coordinate system of the scanner (xs,ys,zs) can be considered fixed while the head coordinate system (xh,yhizh) can be thought of as variable. For this research, MR images had dimensions O.7xO.ï x6.6 mm/voxel and SPECT images had 10 x lOx 10 mm/voxel resolution dong the X, Y, and Z axis respect ively. To solve the regist rat ion problem, a transformation t hat maps one image set into the other was sought. Since the brain doesn't deform or move appreciably inside the skull, the transformation desired is a translation + rotation + scaling corn- bination. It doesn't include anisotropic scaling because that would only be caused by organ deformation. Generally speaking, to perform regist ration one has to:

Figure 3.5: Orientations and spacings of the section planes in different machines.

rescale the data set having poorer resolution (SPECT in our case) to bring it to the sarne dimensions in pixels as the set with better resolution (MRI in our case), "reslice" the rescaled SPECT data in the same planes and at same resolution as the MRI data, and superimpose corresponding slices of the MRI data and the transformed SPECT data to obtain resulting images that contain information from both sets.

Fig. 2.6 shows, in two dimensions, the concept underlying this solution to the registration problem. Figure 2.6: Steps for registration.

This research considers only the first two steps, namely, rescaling and reslicing. Fusion seems to be a representational task which can be achieved fairly easily. An example implementation of fusion will be discussed further in section 5.2.

2.2 The Absolute Orientation Problem

There are several approaches to solving the registration problem. Usudly every particular problem imposes certain constraints and thereby limits the methods which may be used. The simplest solution to our problem is, probably, tc fix the patient's head inside the machine in a unique way so as to assure that the imaging planes are consistent and rigidly defined anatomically. Needless to Say, that this could be very uncornfort able psychologically even if it doesn't hurt him or her physically. Anot her way to deal with this problem is to attach sensors to the patient's head in a unique pattern and to position the patient's head interactively by aligning the sensor array with some template. This requires alterations to the existing hardwâre as well as the involvement of trained staff. An alternative approach is to design software or a system including hardware and software that is capable of bringing together sets of images that have different orientation of their section planes. The last ap~roachis t &en in t his research. Al1 the methods that require computations to perfotm matching are characterized as computationai methods. There are many applications outside the medical imaging area where the problem of registration appears. All of them consider the same problem of fitting the given (rotated, translated and scaled) data pattern to mode1 patterns. .A simple example is the matching of an elevation map created le cdly to the larger range map of the Earth's surface collected by a satellite. Every pattern recognition application in which there is no unknown anisotropic scaling faces the same problem of matching the given data pattern to the set of pre-stored mode1 patterns. An assumption was made in this rese~chthat no anisotropic scaling takes place. The problem of finding the correspondence between two patterns in m-dimensional space is sometimes called the absolute orientation problem. Patterns to be compared can be represented in different forms such as in implicit or parametric form. The point-set representation is of particular interest. A point set is a collection of points belonging to the given pattern represented by their coordinates in m-dimensional space (131. If the surface or pattern is represented in some other way it can be transformed into point-set form by quantization. Using point set notation, the absolute orientation problem can be stated as follows: if {xi) is a data point set in m-dimensiond space and {yi) is a mode1 point set, then the transformation which maps {xi) to {yi) is sought. The case of registering MRI and SPECT data requires a solution of the 3-dimensional absolute orientation problem. These MRI and SPECT data are inherently in point set notation. To register two sets of bdimensional data properly, it is necessary to find the optimal rotation: translation and scaiing that bring one point set to the orientation, location, and scale of the other. The computational methods used to solve the absolute orientation problem cm be divided into two major classes according to the nature of the data in the particular problem to be solved. One class includes al the methods that use raw data for the purpose of rnatching. The above mentioned example of matching an elevation map to the range map of the Earth surface is an example of this class. Elevations of a patch of the Earth surface collected locally are similar in nature to the data gathered by the satellite. Therefore, the point sets to be compared can be forrned directly from the elevation and range data. In this first class of cornputational rnethods, one of the most general approaches to the problem of solving the absolute orientation probiem was suggested by Besl and McKay [20]. Their method solves the problem of registration between two point sets that belong to the same surface or curve. One of the sets is rotated and translated. They proposed the ICP (iterative closest point) algorithm that globally minimizes the mean square distance error between the two point sets over al1 six degrees of freedom. Their method uses quaternion mathematics to describe rotations of a rigid body in 3-D space. The authors achieved excellent results in registering point setst spatial curves and 3-D surfaces even when the data were contaminated wit h gaussian position noise with standard deviation equal to 10% of the size of matched objects. Their algori t hm converges monotonically to the nearest local minimum of a mean-square distance metric, and the rate of convergence is fast during the first few iterations. This algorithm is good for finding a match to a given spatial curve or surface from a set of curves or surfaces having vaxious orientations and shapes. Unfortunately, this rnethod cannot be applied directly to SPECT-teMRI registration for two reasons. Firstly, though both data sets to be registered are taken from the same physical object (the head) the techniques used for acquiring the data differ greatly. Areas of hi& activity that show up in a SPECT image can include several different structures in the brain. Thus, there is lit tIe Likelihood that a match for a particular structure found in the MRI data can be found among the structures from the SPECT set because the two imaging techniques present different information. Secondly, the data acquired in both imaging modalities is volume data. To implement this algorithm, some surface present in both data sets (such as the surface of the brain) must be sought. This would be a source of significant error especially when dealing with SPECT data which have low resolution. Methods constituting the other class require the extraction of " key" features for registration. Usually these techniques are employed with volume data rather than with surface data. Since data received from any source contains both useful information and some noise, it is necessary to extract some features or landmarks from the raw data. This class of methods depends on the assurnption that the transformation between corresponding points in the data sets varies srnoothly or at least in a marmer indicated by the transformation between the corresponding representat ions of the key features. Mat hematical met hods used by " key feat ure" regist rat ion for finding the actual transformation can be the same as those used by the first class. MR image SPECT image

Figure 2.7: Corresponding MRI and SPECT slices.

Key feature registration can be divided in two subclasses according to the nature of the landmarks used for determination of the transformation. The first subclass of methods uses natural landmarks that are present in both sets. This approach is not appropriate for SPECT-to-MRI registration because of the poor resolut ion of SPECT imaging. Not only would it result in big registration errors. but even detection of landmarks in both data sets is likely to be problematic. These difficulties are illustrated in Fig. 2.7 showing corresponding slices of a phantom study conducted on both MRI and SPECT machines. As can be seen, it is difficult to accurately define corresponding structures in MRI and SPECT sets due to the nature of the images and to the low resolution of the SPECT image. The other possible way of registering data is known as the "hat and head" method [4]. Knowing the position of the easy-visible visible " hat" permits estimation of the position of the not-sevisible "head". Thus, this method requires using an externai reference system of fiduciai markers that can be attached to the skull. This reference system provides a set of artificial landmarks for determining the t ransfor- mation. Use of an external frarne is inconvenient but it provides a reliable source of information about the orientation of the patient's head inside the imaging system. Computat ional rnethods for solving the absolute orientation problem can also be classified on the bais of the way the algorithm works. Some algorithms are iterative while others are non-iterative. Besl and McKay in (11 suggest an iterative algorithm for finding the registration transformation. There exist a number of standard optimization techniques that cm be used in the implementation of these met hods such as Powell's direction-set met hod, the Neider-Mead downhill simplex method, simulated amealing, steepest descent, Newton's method, or conjugate gra- dient method [3]. The generd idea of an iterative solution to the absolute orientation problem is to compute at every iteration the value of the registration error that is to be minimized and to modify the transformation so as to reduce t his error until it is acceptable. Iterative methods are applied most successfully when it is necessary to match data that axe arranged in an unknown way. This may be seen in the following example. Suppose that two triangles must be matched (Fig. 2.5 ). For simplicity they are assurned to be congruent. Suppose that there are two possible ways of taking measurements: at designated points (say at the vertices and at the mid-points of the sides as in Fig. 2.9 ) or at arbitrary points as in Fig. 2.10.

Figure 2.8: Problem: match triangles.

Figure 2.9: Reference points, known positions.

Circles in Fig. 2.9 and Fig. 2.10 represent the margin of error about each measurement that is inherent to the method of measurement (combination of systematic and Figure 2.10: Reference points, unknown positions. random error). If approximate positions of the points where the measurements were made are known a priori (Fig. 2.9), then non-iterative methods can be used. In Fig. 2.9 it is known that, for example, vertex A of the left triangle corresponds to vertex A' of the right triangle. Therefore, with ail corresponding pairs in the two point sets known, the transformation that maps one point set into the other point set in one step can be determined. If, on the other hand, measurements are made at random locations, and it is only known that the measured points belong to the pattern, then iterative methods will produce better results. Indeed, this latter case is a more complex problem of finding the transformation yielding the closest match. In geaeral iterative methods cm be used to solve problems that do not employ feature extraction and in problems in which the order andfor locations of data points are not known. Consider again the exarnple of matching elevation maps. Lo- cal measurements in this exarnple are most unlikely to coincide with the locations of the satellite measurements and so an iterative algorithm that minimizes the distance between the mode1 pattern and the transformed data pattern should be applied. Fea- ture extraction does not rule out using iterative methods, but generdly the required features are knowo. This prior knowledge is what makes using non-iterative methods possible. Fig. 2.1 1 is a tree diagram for the methods for solving the absolute orientation problem.

2.3 Descriptions of Rigid Rotation

There are three mathematical approaches to describing the rotation of a rigid body in 3-D space found in the literature. These are quaternion representation, singular 1 Computational methods of solving absolute orientation problern I

Methods hat use

Methods that do not use

Figure 2.11: Tree diagram illustrating the classification of the computational methods for solving the absolute orientation problern. value decomposition (SVD), and polar decomposition. Al1 these approaches seek the solution for the rotational part of the registration transformation in the form of a rotation matrix. Assuming that translation and scaling are already accomplished. the rotation matrix R, which is 3 x 3 for 3-D space, is multipied by the column vector Zi from the first data set to yield the corresponding vector fiin the second set. Thus,

In equation (2.2) the cross covariance matrix of the two point sets is used to determine the components of the rotation matrix, where pz and F,, are the "centers of gravityn or the first moments of the (5,) and (6)point sets respectively. Al1 the algorithms that use the cross cornriance matrix assume that:

a point sets have the same number of points,

the points in the two sets have identical ordering, and

corresponding points in the two point sets must correspond to the same position in the physical spzce.

The first condition reflects the r-quirzxicnt f~ithe ciuss covariance matrix to be square. The second condition assumcs the same direction of going through the point set in order to rule out the error associated with mistaking "flips" for rotations (Fig. 3.3). These two conditions are relatively easy to satisfy. The third condition requires precise calculation of the coordinates of designated features. This is the sit- uation shown in Fig. 2.9. If the third condition is not satisfied, algorithms that use the cross covariance matrix cannot find the optimal rotation. To overcome this difficulty, a new method is suggested which computes coordinates of designated features from multiple measurements and t hen uses quaternion mat hemat ics to calculate the rotation in 3-D space. Chapter 3

Theory and Implementation of the New Method

In this chapter, design of a prototype reference system is described. Preliminary studies on MRI and SPECT machines are discussed dong with problems that arose during these studies. The general problem of finding the registration transform is then split into several lesser tasks. These include determinat ion of reference system orientation from the reference spots, finding the rotation, translation, and scaling transformât ions, and multilinear interpolation of the result ing " registered" data set.

3.1 Non-iterative Direct Method

As we are dealing wit h what is actually volume data, it was decided to develop an algorithm to implement a feature-extraction method. Some landmarks that are reliably present in both sets of images had to be selected. Because of the poor resolution of SPECT images, an external reference system of fiducial markers is employed. These artificial landmarks are used to determine the registration transformation. An algorithm using this technique, known as a " hat and head" method. extracts well defined fiducial markers in both sets of images, finds the relative transformation of one set with respect to the other from the orientations of the "hatn and then applies the transformation. When the transformation is applied to images of a given rnodality, a new scaled, rotated, and translated set of images is created. Then images from this new set are fused with images acquired using the other imaging modality. It was decided to use a reference system consisting of two triangles as a prototype. The position of a rigid body in 3-D space may be uniquely determined from the position of two non-pardel lines belonging to this body. Therefore, any two sides of a triangle cm serve this purpose. It is sufficient to use just one triangle for reference. The second triangle can be used to assess the accuracy of registration. For an imaging session, the reference system must be attached to the patient's skull in such way that the planes of the triangles are approximately parallel to the body axis and the head of the patient is located between the triangle planes. Isosceles right triangles (Fig. 3.1) were chosen to equalize imaging conditions between coronal and transaxial orientations. Each triangle is fitted with tubing, filled with a radioactive agent, around its perimeter. As the sides of triangles are "visible" to scanners due to the tubing, they will produce bright spots in the images. Orientation of each side can be deduced from the positions of the spots in multiple images.

Figure 3.1: Structure of triangles.

The sides of the triangles are therefore the features of interest to be extracted in both sets of images. With this knowledge about the shape of the reference system. it is possible to directly solve the absolute orientation problem. The triangles must be positioned so that several MRI and SPECT section planes will intersect them. As the position of a rigid body is uniquely determined by the position of two non-parallel lines belonging to the body (the sides of triangles), it is sufficient to use just one triangle for the reference. As is shown in Fig. 3.2, normalized vectors that have the same orientation as the triangle sides can be used. Here

r vector VI corresponds to the line AB, vector V2 corresponds to the Lne AC, MRI SPECT

Figure 3.2: Unique position of a rigid body.

0 vector V; corresponds to the line A'B', and

0 vector V; corresponds to the line .TCf.

To find the rotational part of the registration transformation, two corresponding pairs of normalized vectors are analyzed to find the combination R of rotations that transforms one pair into the ot her. To complete the determination of the registration transformation, the translation rand scaling c have to be found. In determination of rotation, the orientation, and not the location of the vectors is of interest. This justifies the use of normalized vectors. If the vertex '4 is assumed to be the center of rotation, then to find translation vector ,4X must be found. This means that the coordinates of this vertex are to be found in each data set. The Euler theorem states that any two states of a rigid body with a single fixed point are separated by oniy one rotation about sorne axiç. To find this axis and the angle of the rotation, quaternion mathematics was selected because it fits this class of bdimensional problems extremely well. The idea of the suggested "hat and head" algorithm can be described as follows: a find the transformation that maps the reference systern from one image set to the other, and

a apply this transformation to every voxel of one of the sets to get the position of the corresponding point in the other set.

The assumptions underlying this algorithm must be explored. First, the current configuration of the reference systern does not allow the use of sagittal images. Only coronal and transaxial sets may be used. This drawback cm be eliminated by adding a third triangle in the transaxial plane. Second, as the triangle used for the reference is isosceles, the range of rotations that can be distinguished is limited. For example, it is impossible to Say whether the reference triangle is flipped or rotated in the plane in Fig. 3.3. In the sense of unconstrained rotation an ideal reference system would be a 3-D star of straight lines al1 passing through one point and al1 having distinct appearance.

C i?'

Figure 3.3: Left to right: initial position, rotated position, flipped position.

For the current application ( SPECT-to-MRI registration) the limi ted free space in the imaging systems means that the angular position of the patient's head in one machine cannot differ by more than about 30 degrees from its position in the other machine. This forces al1 the possible orientations to lie within a solid angle of 30 degrees (Fig. 3.4). Note that though human anatomy allows greater angles (especidly about the principal axis), they are highly unlikely in t his application. If the solid angle of possible orientations was greater than 90 degrees, the use of isosceles triangles would not be applicable. In such a case a different type of triangle should be used, for example, a right angle triangle with angles of 30 and 60 degrees. The spatial constra.int of the imaging machines ensures that reference points belonging to the correct side of the triangle will be considered and the correct rotation matrix, not a reflection of it, will be computed. Another consequence of this position Figure 3.4: Solid angle of plausible rotations.

constraint is that, for example, a coronal data set in one machine cannot possibly correspond to a transaxial or sagittal data set in the other scanner. The registration algorithm for the reference system using triangles now can be described as follows:

in both sets find orientation and position of the chosen triangle,

0 find the rotational and translational parts of the registration transformation using two pairs of corresponding vectors.

0 find the scaling part of the registration transformation. and

0 apply the registration transformation to every voxel in the MRI set.

Note that the set of images having higher resolution (hlRI set) is mapped into the SPECT set. If the SPECT data were mapped to the MRI data, some information would be lost due to aliasing. By making the triangle of sufficient size, at least two slices are guaranteed to intersect it. It is not necessary for the detected points in the two data sets to correspond to the same physicd position since the algorithm does not cornpute the cross covariance rnatrix. It is merely sufficient for them to belong to the same side of the triangle. Thus, the problem of rigid correspondence between the points in two point sets is solved by using knowledge about the geometric shape of the pattern to be extracted from the data. Another advantage of the new method is that as few as two reference points per side of the triangle are required. Increasing the number of detected reference points will, however, improve accuracy. This aspect will be studied further in section 3.7.1.

3.2 Revisiting Methods Based on the SVD

Different mat hemat ical met hods of finding the regist ration transformation were discussed in section 2.2. Among the non-iterative methods, one that uses the singular value decomposition (SVD) of the cross-covariance matrix appears to be most applicable. This method determines the rotation, translation, scaling and rnean-square regist rat ion error. The problem, as stated earlier, is that often corresponding points in two sets do not refer to the same location in the object. This can be caused by noise that is present in the data set and/or in the mode1 set, by the randomness of measurements or by both of these factors. Historically, the first algorithms assumed that noise was present only in one data set. .4n example of this SVD-based algorithm is described by Kim [12]. This algorithm was not guaranteed to return a rotation matrix, and might instead return a reflection matrix. Umeyama [%6]improved the algorithm so that it always returns a rotation matrix. Later Goryn and Hein made further improvements [5] so that both sets could be corrupted by noise. This method finds the cross-covariance matrix of rotation, translation, scaling and mean-square registration error. In [5] the following theorem is proven:

Theorem 1 Let X = XI,X*, .... x, and Y = yi, y2, ....y, be corresponding point patterns in m-dimensionai space. The minimum value c2 of the mean squared emr

of these two point patterns wzth respect to the similan'ty transformation parameters (R: rotation, t: translation, and c: scaiing) is giuen as follows: where

and let a singular ualue decomposition d2 2 ... 2 dm 2 O), and

C, is a covariance matriz of X and Y, pz and py are mean vectors of .Y and Y, and oz2 and ou2are variances around the mean vectors of X and Y. respectively. When mnk (x,,) > rn - 1, the optimum transfomation parameters are determined uniquely as follows:

where S must be chosen as

when (rank(C,,) = m - 1)

According tn equations ( 3.9-3.10) to find the parameters of transformation it is suffiiient to fiod the cross-covâriauce matrix x, and perform the SVD for this matrix. The elegancc and simplicity of this method have made it very popular. It was shown in section 2.3 that computational methods that use the cross- covariance rnatrix rely heavily on the order of the points in corresponding patterns. This is because the coordinates of the centers of gravity for the patterns must be computed. Because of the smdl number of points, different locations and different spacing between points in patterns in SPECT-bMRI registration, it is difficult at best to apply this method directly to the acquired data (Fig. 3.5).

-0 w / A correspondence I / / I / / I L,,,,,,,,-,-,,,-,J

Figure 3.5: Acquired data.

To make the SVD method work, at least three pairs of corresponding points must be defined. An ideal reference systern therefore would consist of three point sources situated at the vertices of a triangle. However, it would be difficult to determine their exact locations in the SPECT images. That leaves us with the necessity of using a very large object as a reference systern. The coordinates of the vertices could then be cornputedo using the line fitting technique described in section 3.7.1 for lines (AoA'), (B, Br),and (C, Cf)in Fig. 3.6. There is an obvious ~roblernassociated with this solution: no rnatter what orientation is chosen - coronal, transaxial, or sagittal, there will always be a side of the triangle t hat intersects the imaging plane at an acute angle. This rnakes computation

Figure 3.6: Difficulty in using of vertices as reference. of the coordinates of two vertices from a single projection unreliable (Fig. 3.6). Use of a second projection, (for example, a transaxiai projection) in addition to coronal projection, will require registration between projections. Thus, a solution involving a second projection may also be computationally expensive because the amount of data to be processed is doubled.

3.3 Reference System

As it was decided to follow the " hat and head" method, a reference system was required. MRI technology requires that no ferromagnetic parts be used in the reference system. Accordingly, the reference system is made out of plexiglass (Fig. 3.7) and consists of a head piece and four detachable triangular frames. The headpiece is mounted on the head of a patient in a unique way. Positioning of the headpiece is provided by five points of contact with the patient, two of which are adjustable. Non- adjustable points provide support of the reference system at the bridge of the nose (A) and two points on the upper part of the skull (B and C). Adjustable points D and E are the "earplugs" each having three degrees of freedom to fit into the meatus of any patient. The frames hold isosceles right-angle triangles that have tubing running around their perimeter. One set of triangles is used for each method. Triangles are removable from the frame while the frame is on the patient. The sides of each triangle are 95, 95, and 134 mm respectively. The tubing is typically filled with a radioactive agent that can be seen in images acquired from both WRI and SPECT machines. For MRI- only studies, tubing can be filled with different substances such as an organic oil that produces sufficiently bright marks in images. After points D and E axe adjusted for the patient, the reference system can be removed by undoing bolt F without disturbing the adjustment of points D and E. It may be remounted by loosening bolt F, fitting it onto patient's head, and then retight- ening the bolt. Since the triangles contain a radioactive agent that fills the perirneter tubing, it is preferable to mount the triangles after adjusting and reposit ioning oper- ations are complete, irnmediately before scanning.The current setup allows imaging at two of three possible main orientations: coronal and transaxial. The possibility of modifying the system for sagittal imaging is discussed in Chapter 5. triangle

Figure 3.7: Reference system prototype. 3.4 Preliminary Studies

For reasons of convenience it was decided to conduct al1 the experiments using a coronal orientation. Preliminary studies had as t heir goal the determination of the optimal diameter of tubing and the right concentration of radioactive agent. The reference system was scanned in both MRI and SPECT machines. There were several problems that became apparent after preliminary MRI and SPECT scans of the reference system:

a The height of the triangle in the z direction may be less than the total span of the data set in the z direction. This leads to the absence of reference points in marginal slices (Fig. 3.8(a)).

0 Because of the spacing between images in most cases, it is hard to determine the exact location of the triangle vertices directly from one of the slices (Fig. 3.8(b)). The triangle can protrude beyond the marginal slice so that the vertex of the triangle does not belong to the image set. (Fig. 3.S(c)). The tubing does not consist of just three straight segments. It describes a circular arc at each vertex. This can cause problems in finding the correct vertex coordinates if the slice in question includes this arc. (Fig. 3.8(d)).

Thus, the algorithm to be designed has to deal with:

Absence of any reference points in the slice t hat is maiyzed (Fig. 3.8(a) and (b) Incorrect determination of the reference point location due to the curvature of the tubing (Fig. 3.S(d))

Clearly, these problems appear in marginal slices. Thus, the algorithm, first discard al1 the invalid slices, keeping those containing two reference points per triangle. To overcome problems with tubing arcs at triangle vertices, the reference system was modified. Specid markers were attached to every vertex of each triangle (Fig. 3.9). These marginal markers are short pieces of the tubing wit h larger diameter. Preliminary experiments confirmed that due to the physical constraints im- posed by the lack of free space inside scanners, the orientation of the triangular frarnes of the reference system axe always close to the sagittal plane (the YZ plane in Fig. 2.2). This allows left and right parts of every slice to be considered separately because reference spots from one side of the reference system always appear in the corresponding half of the image. Sometimes the orientation of the reference system inside the MRI scanner is such that in the whole set a given triangle appears in fewer than 4 slices. However, this is not likely to happen simultaneously on both left and right sides. This problem aises only in the MRI set. The volume containing the patient's head in the SPECT machine is much bigger than in the MRI machine. The SPECT imaging volume always contains the cornplete reference system regardless of its orientation.

3.5 Reference System Orientation

It is assumed that the position of the reference system reflects the position of the skull, and hence, the position of the brain inside the skull. Thus to find the desired transformation it is suficient to determine the correspondence between the positions of the reference system in both sets of images. The transformation which inaps the SPECT set to the orient? tion of the MRI set is sought. Figure 3.8: Problems encountered in preliminary studies. (a) Marginal slice is "empty". (b) Vertex lies between slices. (c) Vertex does not belong to the set. (d) Tubing curvature imposes a problem. marginal marker View A

Figure 3.9: Marginal markers added to the reference system.

Due to its shape, the reference system will appear in most slices of both sets as four bright spots outside the region of the patient's head. If pairs of these reference points belong to the same trimgle, they will appear closer to each other for slices closer to the vertex of the triangle (Fig. 3.10). To find the components of the registration transformation, first it is necessary to determine the positions and orientations of the reference triangle in both sets. By computing the coordinates of the reference points in different slices it is possible to recover the orientation of the reference system in space. This operation is done as follows:

Segment the image - try to find two points in every slice that belong to the chosen triangle,

0 Find centers of the reference points.

Connect corresponding centers of the reference points in successive images,

Find the location of the vertex at the intersection of the lines just found.

It then becornes possible to determine the transformation that maps one orientation into another.

3.5.1 Image Segmentation

The operator should decide what side of the the system to take as reference. In future, this part may dso be automated. Once the reference is selected, the remainder of the algori t hm does not require human intervent ion. slice N

Figure 3.10: Location of reference points in images is a function of proximity to the triangle vertex.

First, the algorithm segments every image in both sets. Reference spots are extracted and the coordinates of their centers are found. Since the reference system is designed so that each triangle is Iocated about 30 mm from the patient's head, the reference points may be found by a simple masking operation. The idea of the masking operatioo is to remove the central piece of an image and to keep reference spots. Reference spots are detected as follows:

1. First the original monochrome image (Fig. 3.11(a)) undergoes a low-pass filtering operation. Low-pas filtering is used to achieve two goals.

(a) First , it removes most of the high-frequency noise. (b) Second, low-pass filtering blurs the image and "attaches* those parts of the soft tissue that seem to be disconnected from the head (sometimes ears or nose can produce such an effect).

2. The blurred image is then thresholded at a level corresponding to 10% of the maximum intensity, producing a binary "mask" image (Fig. 3.1 1(b)) For comparison, the result of thresholding performed on the non-blurred image is presented in Fig. 3.1 1(c).

After the first grey-level thresholding, the algorit hm performs area t hresholding by discarding dl the white spots in the binary image with areas less than the lower threshold or greater than the upper threshold. The resulting binary image which is given in Fig. 3.12(a) is then used to mask the original image. Fig. 3.12(b) shows the result of this operation. (4 ('4 (4 Figure 3.11: (a) Original monochrome image. (b) Blurred image thresholded at 10% of the maximum intensity. (c) Original image thresholded at 10% of the maximum intensity.

(4 (b) (4 Figure 3.12: (a) Mask image. (b) Result of masking. (c) Masked image thresholded at 30% of the maximum intensity.

The area thresholds for both MRI and SPECT were determined after examina- tion of the results of grey-level thresholding. They are different for MR and SPECT due to the difference in resolutions. For the MRI the area thresholds equal 2 pixels and 100 pixels for the lower and upper thresholds respectively, while for the SPECT the thresholds equal5 pixels and 50 pixels for the lower and upper thresholds respec- t ively.

3.5.2 Valid versus Invalid Images

The next task for the algorithm is to discriminate between valid and invalid reference spots. For that purpose the masked image (Fig. 3.12(b)) undergoes a second grey- level thresholding at a level corresponding to 30% of the maximum intensity of the masked image. The result is a binory image which is shown in Fig. 3.12(c). This image is analyzed to determine its validity. The following criteria are used to decide whether the slice is valid or invaiid:

there must be exactly two white spots in the reference half of the image.

0 for the two reference spots, the y-coordinate distance between t heir cent roids must be greater t han 5 mm. areas of the white spots are computed by counting the numbers of pixels in both reference white spots. These areas must not be lower than 3 pixels and not greater than 15 pixels for both SPECT and MRI sets.

O dong with the area of each spoto the algorithm assesses its shape. For each spot standard deviations in pixels about the centroid in both X and Y axes are computed. These standard deviations must not exceed 5 mm.

The first three criteria were devised to detect various conditions that can appear in marginal slices. Marginal markers yield bigger spots than the " normaln reference spots. If one of the "normaln spots is not detected, no spots are detected, or at least one of the detected spots is suspiciously big, then the slice is considered invalid due to the first three conditions. The fourth condition is added to detect special cases in which the tubing describes an arc or it crosses the slice plane at an acute angle. Elongated reference spots characteristic of these cases provide an unreliable source of information for accurate calculation of their centroids.

3.6 Determination of Scale

The ultimate goal of the algorithm suggested in this thesis is to produce a SPECT data set wit h the dimensions, resolution and orientation of the MRI data set. Image dimensions in the SPECT data set are 64 x 64 x64 voxels while for the MRI data sets used in this research the dimensions are 256 x 256 x23 voxels. Bringing sets to one size does not mean just simple quadrupling of the SPECT images because actual data can occupy different volumes inside the set. Suppose that the orientations of two data sets are the sarne and two images from these data sets are brought to the sarne visual scale. In our case, in SPECT images a brain phantom is about 20 pixels wide or 113 of the image, compared to 130 ?ixels or 112 of the image for MR images. Thus, simple enlarging of the whole SPECT image to the dimensions of MR image will not result ir the correct scale. 3.6.1 Volume Rendering

This thesis proposes the use of a modification of the direct volume rendering (DVR) technique [14] to solve the scde problem. DVR is used in visualization of 3-D data. The proposed modification is to consider the spatial coordinates of points in both sets in millimeters rather than referring to points in the data sets in tem of pixels. By "normalizing" both sets in this way problems with different spatial resolutions of the sets are avoided. The spat id coordinates of each pixel are determined by multiplying the values of its coordinates by the resolution dong the corresponding axis. After the image is segmented it is easy to find the x and y coordinates of the center of any reference spot by determining the means of the x and y coordinates of al1 of its pixels. These computations are performed using floating point arit hmetic. Knowing the number of slices from the beginning of the set and the resolution along the Z axis (the thickness of the slice), the z coordinate of the reference point's center may be cornputed. Slices have some thickness i-e. it is volume that is digitized. Tubing that is nonperpendicular to the plane of a slice appears in images not as a circle but rather as a figure consisting of two elliptic and two straight segments (see Fig. 3.13) It is assumed that the center of a reference point represents the point that lies closest to the mid-line of the tubing, and thus cm be used for determining the orientation of the side. (Fig. 3.13 ). -tubing

image l I I

Figure 3.13: Close-up of the tubing crossed by a slice. 3.7 Det ermination of Translation and Rotation

For an MRI data set, the assumption that the centers of two reference spots can be used to determine the orientation of the vector collinear to the side of the triangle corresponding to these spots is valid due to the high resolution of MR imaging technology. To find the vector orientation it is suEcient to deterrnine coordinates of the centroids of the reference points in any two slices, preferably as far apart as possible. However, the poor resolution of SPECT imaging can cause registration errors due to incorrect detemiinat ion of the vector's orientation. Usually t here are more t han two valid slices in any set. This raises the possibility of using this redundancy to minimize the registration error. This can be done by fitting a straight line to the "cloud" of points in 3D space that belong to a particular side of the triangle and assuming that this straight line has the orientation of the side.

3.7.1 Fitting a straight Line to a 3-D "Cloudn

The task is to find an analytical expression for the 3-D line. To do this, it is sufficient to find expressions for its projections onto the orthogonal planes. The method used in t his algorithm performs a least-squares estimation of the parameters a and 6 of the general equation of the line in 2-D space [23]:

Coefficients a and 6 are estimated as: where

â and 8 are estirnates of a and 6, respectively rn, and rn, are the means of the sequences {xç) and

Two straight lines are to be found using this method: r = f(r) and y = (z). This least-squares fitting technique is not applicable for the "clouds" that are stretched in a direction that is orthogonal to r. Estimate â can be determined incorrectly in this case. Let us assume a "cloud" which orientation axis is f15 degrees orthogonal to z to be invalid. However, the limitations and assumptions given in section 3.1 forbid such an orientation of the "cloud" . In the worst case the orientation of the " cloud" will constitute 45+30/2=60 degrees to the z axis or 30 degrees to any axis orthogonal to r . This is two times larger than the assumed tolerance. Therefore, the least-squares fit ting procedure can be employed. Let us consider a set of points representing the centers of the reference spots in three-dimensional space. By assuming independence of the coordinates s and y, two straight lines may be fitted to two point sets in perpendicular planes XZ and YZ. Planes b and c that contain projections B and C of the line A are orthogonal to planes YZ and XZ respectively (Fig. 3.14). By using projections, the 3-D line-fitting problem cm be reduced to its 2-D case. Coordinates of the reference points for the whole data set can be arranged in sequence. Table 3.1 represents such a sequence obtained from an actual data set. The first column contains the number of each slice, columns 2-5 hold the coordinates of two reference points in the left hdf of the image, columns 6-9 hold the coordinates of two reference points in the right half. Zeros in rows that correspond to particular slices mean that the segmentation algorithm has found these slices to be invalid. The straight lines are fitted to the sequences {xk(z)}and {yk(r)), both of N points. Line A projects to line B on the YZ plane and the line C on the xZ plane. Points along the lines have x and y coordinates given in sequences {xk) and {yk}. The z coordinates of the reference points are at constant intervals because slices have uniform spacing. They are the product of the slice number and the resolution in the z direction. The fitting algorithm consists of the following steps: Table 3.1: Example of coordinates (in pixels) of reference points for an MRI data set. Figure 3.14: Reduction of the 3-D Iine-fitting problem to 2-D cases in planes YZ and

0 Find the coefficients for two projections using the({xk},{zr}) and ({yk},{zk)) pairs of sequences.

0 Using projections, find "refined" coordinates at which the 3-D line intersects the first and the last valid s1' lces.

"Refinedn coordinates x-f irst and y-first are cornputed by substituting the z- coordinate of the first valid slice in two equations of the form of equation (3.11). 'Re- fined" coordinates xlasf and ylast are cornputed by substituting the z-coordinate of the last valid slice in these two equations. Two "refined" points determine the orientation of the fitted straight line in 3-D space (Fig.3.15). This technique for fitting a straight line to a set of points in 3-D space results in increased accuracy in determining the orientation of the triangles' sides. As more vdid slices are present in the data set, the more accurate the calculation of the orientation of the side of the triangle becomes. Line fitting is performed for two sides of the triangle. The orientation and position of these fitted lines are then used for determining the translational and rotational parts of the registration transformation. Part

After finding the refined positions of the vectors representing the sides of the triangle in both sets, the next step is to find the coordinates of the vertices - the points where the two lines representing the sides of the triangle intersect. This is needed to determine the translational part of the transformation which is manifested as a displacement between two corresponding vertices. It is obvious though, t hat despi te the refinement process, lines representing the sides of the triangle do not meet at one point in space. This happens because of quantization error and computational errors while determining the centers of the reference points in the slices. The coordinates of the vertex of the triangle must be computed. Wit h reference to Fig. 3.15. location of the vertex dong the Z axis is estimated as the location of the rniddle point of the line ST. Line ST intersects both fitted straight lines and is orthogonal to them (Fig. 3.15). Once the coordinates of the corresponding vertices are cornputed for both sets, it becomes possible to determine the translat ional part of the registrat ion transformation. This translation is calculated as a vector connecting the corresponding vertices (equation 3.14).

where Xmr = x-coordinate of the vertex in the ,MRI set,

Ymr = y-coordinate of the vertex in the MRI set, Zmr = r-coordinate of the vertex in the MRi set,

xsp = x-coordinate of the vertex in the SPECT set, YSP = y-coordinate of the vertex in the SPECT set, zsP = z-coordinate of the vertex in the SPECT set, 44- , k = triple defining the Cartesian coordinate system.

3.7.3 Rotational Part

With the aid of quaternion algebra, finite rotations in space may be dealt with in a simple and elegant manner (see appendix A). The application of quaternion algebra to SPECT-to-MRI registration is based on the fundarnental theorem given in [15]

Theorem 2 If q and r are any non-scalur quaternions: then vectors

Figure 3.15: Vectors represent ing sides.

is a quaternion whose nom and scalar are the same as for r. The tlector is obtained by reuolving conically about through tvice the angle of q with respect to r.(Fig. 3.16)

Figure 3.16: Rotation in terms of quaternions. is obtained by revolving conically about ë through an angle 2Q. The norrndized quaternion has .Plq = 1. Equation ( 3.16) for a normalized quaternion is simplified to:

Consider now the special case when vectors and lie in the plane of rotation. Knowing vectors and ? allows the angle between thern and t he orientation of vector ë to be computed. Vector ë is simply their vector product. Thus, a quaternion can be defined using a pair of vectors. Ncw a succession of rotations must be defined in terms of quaternions. Since the product of quaternions is also a quaternion (see appendix .4) then any number of rotations is equivrtlent to a single rotation. The succession of two rotations defined by quaternions p and q respectively corresponds to the operator

To find vectors representing the sides the triangle the results of the fitting operation are used. According to Fig. 3.15, the beginnings and ends of these vectors are found as the points of intersection of the fitted straight lines with the last and the first valid slices. There are two pairs of vectors VI, 4':h, and V;. Vector I/; corresponds to y':', and vector corresponds to q.The task at hand is to find a normalized quaternion given two pairs of corresponding vectors. The solution is sought in the form of two successive rotations. Quaternions representing rotations cmonly be computed if corresponding vectors have same length. To satisfy this requirement the longest vector is sought and three other vectors are scaled up to have the same length. The solution to the problem is shown in Fig. 3.17. Figure 3.17: Finding the rotational part of the registration transformation as a combination of two rotations. (a) First rotation. (b) Second rotation.

The first rotation is a flat rotation that maps I/I into 4'.The axis of this rotation coincides with vector product V,, * V;'. According to equation (3.17) the normdized quaternion that represents this rotation can be written as:

The first rotation results in going to hjnt.Now the second unit quaternion q2 that rotates hintinto V; conicdy about axis V; can be calculated. As for the first quaternion a couple of vectors that lie in the plane of rotation should be used. The vectors PGint md PV; that are projections of hintand on the plane of rotation are used. Quaternion q, that rotates hintinto V; conically also rotates PVZintinto Pbr and cm be determined from them. Projection vectors Pbet and PV; are found as: Unit quaternion q2 is found according to (3.17) as:

Finally, acc~rdingto equation (3.18)?multiplication of quaternion qz by quaternion ql results in a quaternion q that represents rotation of vectors VI and V2 into V; and V; respectively. Quaternion q is also a unit quaternion because it is the product of unit quaternions qi and 92. Once this resultant quaternion q is computed the rotation is defined.

3.8 Applying the Registrat ion Transformation.

As wns noted in section 3.1, it is more meaningful to process the data set with the lower resolution. Though the reverse approach is possible it will lead to the loss of useful information. In Our case the SPECT data set has the lower resolütion and it has to be "reslicedn dong the planes in which slices of the MRI data set lie. The registration is done voxelwise. A vector is found that connects the vertex of the reference triangle and the center of the voxel under consideration in the MRI data set. The registratioo transformation in the form of the quaternion operator q(-)q-' followed by translation tmay be applied to this vector. According to equation 3-25, this operation yields a vector with coordinates of the same physical point but in the SPECT data set. Figure 3.18: Voxel andysis in an MR image.

Fig. 3.18 depicts an exarnple of the original vector in the MRI set. The final algorithm for voxelwise transformation can be stated as follows. For every voxel in the MRI set:

find the coordinates s.w~~and YMRI of the vector connecting the com~uted vertex A with the analyzed voxel in " MRI voxels" (Fig. 3-18), change units to millimeters using known resolutions in the MRI set: xmres, ymres, and zrnres along the X, Y, and Z axis respectively and number of slices from the beginning of the set NMRI:

tprior= XMRI * xrnres = x-coordinate in millirnetres, = YMRI * ymres = y-coordinate in millimetres, rpior = NMRft zmres = z-coordinate in millimetres,

a find coordinates z,f,,, yajter,md z.fter in millimeters of the vector in the SPECT set by applying the registration transformst ion (equation 3.25), change units to "SPECT voxels" using known resolutions in the SPECT set: xsres, ysres, and zsres along the X, Y, and 2 axis respectively.

XSPECT = xaIter/xsres = z-coordinate in SPECT voxels, ~SPECT= yalt,,/ysres = y-coordinate in SPECT voxcls, ZSpEcT = zafter/~~~e~ = number of dice in SPECT data set, address the SPECT set using SPECT coordinates to find the voxel value (Fig. 3.19 ) ,

0 mite the voxel value to the new, transformed pseudeSPECT data set at the MRI coordinates.

Thus, the correspondences between each MRI voxel and the SPECT voxel are detemined. By applying the transformation to al1 the voxels of the MRI set and finding corresponding voxels of the SPECT set, slices having the orientations of the MRI set are found through the SPECT set.

Figure 3.19: Voxel analysis in a SPECT image.

The accuracy of the method depends on the position of the origin i.e. on the beginning of every vector. Theoretically, any point in space with known position in both sets could be an origin. The vertex of the reference triangle satisfies this condition and is taken as the origin in this research. The algorithm will become less accurate when points that are closer to the origin than the quantization interval are considered because of rounding errors in the computations. For the current research this is not important because the vertex of the triangle lies outside the region of interest.

3.8.1 Multilinear Interpolation

There are two problems associated with the difference in resolutions of the SPECT and MRI data sets.

Several voxels from the MRI set can correspond to the same voxel of the SPECT set. In other words, when being rescaled and resliced, the "registered" SPECT set keeps its low resolution appearance and a prominent "pixelation" effect will be present (Fig. 3.20(a)). SPECT voxeis that correspond to several MRI voxels will be called "supervoxels". Their projections on the image plane will be cded "superpixels" . Even t hough image dimensions in Fig. 3.20(a)are 256 x 256 pixels, inside the superpixels the grey level remains the sarne (Fig. 3.20).

O Being a combination of several images, every image of the new set shows visible edges in areas where the new slice leaves one old slice and enters another (Fig. 3.20). This happens because generally, different slices from the old data set have different intensity levels-

Figure 3.20: Pixelation effect and "edgy" transitions.

Both of these phenornena can be seen in Fig. 3.20. Explanation for these effects is given in Fig. 3.21. In this exarnple we assume a 3-D data set consisting of three layers with grey-level intensities 10, 30, and 20 respectively. A new slice crossing the old data set is shown in Fig. 3.21. Fig. 3.22 shows a one-dimensional cross-section of this new slice by a plane parallel to the axis X. This cross-section consists of t hree homogeneous regions with grey-level intensities 10, 30, and 20 respect ively. Each of these regions or "superpixels" is made of several pixels, which illustrates the " pixelat ion" effect. Transition from one superpixel to another illustrates the "edgy" transitions seen in real data due to different intensity levels of different slices. To overcome the pixelation effect the data set with lower resolution must be brought to the resolution of the other data set. Multi-linear interpolation is used to calculate the variation of grey level inside the supervoxels of the " registeredn data set. Fig. 3.23 is a variation of the Fig. 3.22 with the only difference that the intensities are plotted as a function of distance (in pixels) along the mis X. New SPECT dia (at MM rtsoluîion)

Figure 3.21: Explanation for the pixelation effect and "edgy" transitions.

Figure 3.22: One-dimensional cross-section.

To achieve smooth transitions between supervoxels as well as gradua1 change in grey level within supervoxels it is necessary for every point inside a supervoxel to consider the proximity of the neighbouring supervoxels. The desired resuit of the one-dimensional interpolation for the signal in Fig. 3.23 is given in Fig. 3.24. It was decided to use the total number of six neighbouring supervoxels to determine the value of the new voxel. These neighbouring supervoxels are:

0 "home supervoxel" or supervoxel the new voxel falls inside, supervoxels neighbouring "home supervoxel" along the X and Y axes,

O "stack neighbour" or immediate neighbour in z-direction that lies in succeeding or preceding Lyef depending .a the sign of zbins, and

Q neighbours of the "stack neighbour" along the X and Y axes. Grey levcl

Figure 3.23: Yon-interpolated data.

Grey levcl

Figure 3.24: Goal of interpolation.

As shown in Fig. 3.25, neighbouring supervoxels are depicted by t heir centers. The centers of supervoxels that are assumed to have "influencen are painted dark. To compute the "influence" of neighbouring supervoxels, the algorit hm finds biases xbias, ybias, and zbias at the position of the center of the new voxel relative to the center of the supervoxel it falls inside (Fig. 3.26). Depending on the signs of xbias, ybas, and rbias the algorithm decides what supervoxels it should consider as neighbours. There can be 23 = 8 combinations - the center of a new voxel can fa11 into one of eight octants of 3-D space. Supervoxels cm be defined as "sarne level neighbours" and "next level neighbours". The algorithm treats each "leveln separately. An example of the calculation of the value at point P for one "leveln is given in Fig. 3.27(a). First the grey scale value of the point P in every level is found. This grey scale value is calculated as the mean of the values at points L and M corresponding to the centers of neighbouring supervoxels. In the Fig. 3.27(a) ip = JIL- IM)/2 = 1100 - 101/2 = 45. Then the grey scale value at the point Q is found from the proportion between the lengths of lines KQ and line KP and the grey scale values of points hf and P. For our exarnple, IQ = 1, - IIK - 1p1M =85 -(85 -45)1/3 = 72. next lcvcl x-ncighbour

ncxt levtl neighbours \ next kvel y-ncigfibour

--- --______------

=- new voxel

Figure 3.25: Neighbouring supervoxels.

After the grey scale values of points Q and its counterpart Q' from the next "level" are known the algorithm uses zbias to find gey level value INof the new voxel as illustrated in Fig 3.27(b). Value IN at the new voxel's center is found from the equation (3.26):

ïQ- ~LV--- zbias 1 I - I rres

where IN grey level intensity at the new voxel: IQ grey level intensity at point Q, kt grey level intensity at point Q', zbias bias along the z-direct ion, zres resolution in z-direction. In the figure, if - = 113, and = 42, 721113 = 62. Multi-linear interpolation adds to the information content of each image in the data set because for the computation of each pixel of a "registered" slice it uses information from the neighbouring volume in the old data set. Interpolation is needed even when the voxels in both sets are regular cubes and both sets have the same resolutions because of differences in grey level intensities of pixels in adjacent slices in the old data set. The results of the interpolation algorithm can be seen in Fig. 3.28. infïuences of:

"home" supervoxel

next levei x-neighbour

next levei y-neighbour

\ stack neighbour

/ new voxel

Figure 3.26: Influence of neighbours.

3.9 Implementat ion

Software for the registration algorithm was written in the C programming language. For the purpose of convenience the whole algorithm was divided into two major parts. The first part:

a performs segmentation of every image in a specified input data set.

calculates the coordinates of the reference points, and

a stores the calculated coordinates in the specified file.

The second part:

reads the calculated coordinates from two specified files.

0 computes position and orientation for the reference triangle in two data sets,

computes rotational and translationai parts of the registration transformation,and creates a new, "registeredn data set with new position and orientation. Figure 3.27: Cornputation of the grey scale value of a new voxel. (a) In a "level".(b) Bet ween the " levelsn.

Figure 3.28: Results of interpolation. (a)SPECT image, no interpolation, (b)SPECT image, interpolated. Chapter 4

Experimental Results

The experimentd setup is described in this chapter including the simpie brain phantom used with the reference system. A technique for error measurement is proposed that uses the second triangle of the reference system to estimate the accuracy of the algorithm. Plots of errors as functions of two variables, angle of rotation and distance to the origin, are presented.

4.1 Experimental Setups and Data Sets

SPECT data sets for this research were generously provided by the Department of Nuclear Medicine of the Queen Elizabeth II Health Sciences Centre in Halifax. MRI sets were made at the Radiology Department of the sarne hospital. For both SPECT and MR studies a plastic canister was used as a simple brain phantom. The canister was reinforced with plastic rods to prevent deformation when filled with an appropriate solution. The phantom was also equipped with fillable compartments to imitate structures inside the brain. The compartments were left empty so that their inner volumes would produce no signal and therefore would appear black in the images. It was decided to test the algorithm on images having coronal orientation. For the SPECT scanning procedure the main part of the phantom was filled with a solution containing radioactive Tc(technetiurn). The tubing of the reference system was also filled with a radioactive solution but of higher concentration. Three SPECT data sets with different orientations were acquired. For MR scanning the second pair of reference triangles had their tubing filled with cod liver oil which gives bright spots in the images. The main cornpartment of the phantom was filled with the radioactive solution. A total of eight data sets with different orientations were acquired. Three of those data sets contain phantom studies, the other five data sets are the studies of a volunteer.

4.2 Strategy for Accuracy Estimation

To test the accuracy, two data sets are needed which contain studies of the same object. They will be refered to as the "benchmark" set and the "initial" set. The "registered" set is composed of the data contained in the "initial" set but has the orientation and the resolution of the " benchmark" set. Ideally, the " registeredn set should be identical to the "benchmark" set. Nevertheless, there will be certain differences. The task is to measure these differences. The compartments of the brain phantom were made big enough to be seen in both MR and SPECT data sets. However, accurate measurement of their parameters is difficult, especially in the SPECT set due to its low resolution (see Fig 2.7). The reference system consisted of two triangles. -4ssuming that the reference system does not move with respect to the phantom or with respect to the skull (which is the basic assumpt ion for the whole method), it is possible to assess the performance of the algori t hm by rneasuring the mat ching accuracy of the triangle w hich is not used as a reference for the registration. The triangle which is used as a reference will be called the " reference triangle" while the other triangle will be called an the " idle triangle". One way to evaluate accuracy is to compute the location of the vertex of the "idle triangle" and the orientation of its sides using the same line fitting technique as for registration. After these data are obtained for the 'registered" set they can be compared to the corresponding data from the " benchmark' set. The drawback of this approach is that by fitting a straight line to a set of points the accuracy can only be estimated in a mean square sense. The other way is to compare coordinates of reference spots directly. By looking at the difference in the positions of reference spots, for example, the maximum deviation cmbe found. The behaviour of the algorithm with respect to the parameters of the transformation such as angle of rotation or distance to the center of rotation can dso be traced. Reference spots will behave similarly to structures of small size in the brain. The strategy for testing the accuracy of the algorithm by making use of the idle triangle should inciude the following steps: O extract the reference spots corresponding to the idle triangle, for both "bench- markn and " registered" sets,

O find the coordinates of the extracted reference spots, and compute the differences in coordinates of corresponding reference spots.

Half of this work is already done since the coordinates of the reference spots for the "benchmark" set have been found for the purpose of registration. It is only needed to repeat this process for the "registered" set. The differences in coordinates serve as a measure of the performance of the dgorithm. An example is given in Table 4.1. Obviously, it was required that corresponding coordinates both be valid (non-zero). The algorit hm for accuracy estimation was programmed in the C language. The code is included in appendix B. the algorithm was designed to work with data sets frorn the same imaging modality and with those having different modalities. There are t hree combinations possible:

O SPECT-to-SPECT registration, SPECT-to-MR registration, and

O MR-to-MR registration.

4.3 SPECT-to-SPECT Registration

.An example of application of the algorithm to SPECT-to-SPECT registration is shown in Fig. 4.1.It shows an image obtained by subtraction of the corresponding images from the "benchmark" and "registered" SPECT sets. The difference between the " benchmark" image and the "registered" image that can clearly be seen in Fig. 4.1 has no relation to the performance of the algorithm. Though the two sets are studies of the sarne phantom the difference in their appearance is due to the fact that the concentration of the agent filling the phantom changed from study to study. It is more important that reference points were registered correctly what guarantees the registration of the whole set. Two ways of registration are possible: using one or the other triangle. For convenience the triangles are referred to as "rightn and "leftn triangles. Angles of rotation (in degrees) for the possible cases of registration are given in Table 4.2, where SPI, SPÎ, and SP3 are three SPECT data sets. The angles were cornputed for Table 4.1: Example of a registration error table for an MRI data set. slice &ft~~die t~w C

Table 4.2: Angles of rotation for the SPECT-to-SPECT registration.

regis tration 1 left triangle right triangle angle error Spi -SP2 I 7.5 7.5 .3 SP; - SP~ 11.6 11.1 -5 SP;! - SP3 7.3 7.9 .7

To visudize the error, a 3-D error surface was plotted as a function of the angle of rotation and the distance from the origin (the vertex of the triangle of reference). This surface is shown in Fig. l.%(a). Errors in determining the position of the reference spots of the "reference trianglen were dso computed. These errors axe also a measure of accuracy - they show how precisely the orientation of the reference triangle is determined. Corresponding enor surface is given in Fig. 4.2(b) Figure 4.2: SPECT-to-SPECT registration error surfaces. (a) "Ide triangle". (b) " Reference triangle".

(a) (4 (CI Figure 4.3: SPECT-to-MR registration. (a) Benchmark image. (b) Registered image. (c) Result of subtraction.

4.4 SPECT-to-MR Registration

An example of SPECT to MR registration is given in Fig. 4.3 which shows an image obtained by subtraction of the corresponding images from the "benchmark" MR set and the "registeredn SPECT set. For analysis of the accuracy of the SPECT-teMR registration one data set containing phantom studies was picked among the three acquired using the MR scanner. Registration of three SPECT data sets to this MR data set was performed. Angles of rotation (in degrees) for the six possible cases are given in Table 4.3. The accuracy of the angle computation is .6/2 = f0.3 degrees. To visualize the error the same rnethod as for the SPECT-teSPECT registration was used. E~orsurfaces are given in Fig. 4.4. Table 4.3: Angles of rotation for the SPECT-t*MR registration.

registrat ion left triangle right triangle angle error SPI - MR 5 .fi 5.4 .-3 SP2 - MR 5.3 5.6 -3 SP3 - MR 11.3 11.9 .6

(a)

Figure 4.1: SPECT-teMR registration error surfaces. (a) "Idle triangle". (b) "Ref- erence triangle".

4.5 MR-to-MR Registration

To analyze accuracy of the MR-to-MR registration three brain studies were chosen. It was decided to use real data because the real data sets contain a larger amount of visual information than phantom studies do. This information is in the form of high-spatial-frequency features in the MR brain images which can serve as visual benchmarks in helping to assess the accuracy of the registration. A MR-to-MR registration example is given in Fig. 4.5 which shows an image obtained by subtraction of the corresponding images from the "benchmark" and "registeredn MR sets. The grey level outside the head region corresponds to zero in the grey level scale. The desirable " registered" data set subtracted from the " benchmark" data set would produce zero everywhere. It can be seen that most of the extreme light and dark regions (bad registration) are located not in the brain but outside of it , in the bone and in the neck areas. Note: the images in Fig. 4.5 are stretched in grey level. Thus, these errorc are smaller than they appear. For example, medium grey colour in the "benchmark" image corresponds to the intensity level90 while the same colour in the "errornimage Figure 4.5: MR-to-MR registration. (a) Benchmark image. (b) Registered image. (c) Result of subtraction. corresponds to the intensity level0. Typical statisticd data on the grey level intensity for the region of interest that is located inside the brain area for both "benchmark" and "error" images are given in Table 4.4. The mean of the "error" image is relatively close to zero and its standard deviation is one quarter of the standard deviation of the corresponding region in the "benchmark" image. Range of intensity change is also much less in the "errorn image. Al1 these properties wiiI allow to detect pathological changes in brain structure that will have manifestations in the form of dxk and light areas in the "error" image.

Table 4.4: Statistics on the MR-to-MR registration. I image- II mean 1 deviation 1 min 1 max 1 " benchmark" 77.84 20.68 8 122 error" -4.67 5.13 -22 20

To andyze the accuracy of the MR-t*MR registration three data sets were picked. Angles of rotation for six possible cases are given in Table 4.5 The accuracy of the angle computation for MR-teMR registration is -212 = f0.1 degrees. Error surfaces for the MR-teMR registration are given in Fig. 4.6. Table 4.5: Angles of rotation for the MR-to-MR registration.

registration left triangle right triangle angle error

MRr - MR2 l3.5 13.7 .9-.

Figure 4.6: MR-t-MR registrat ion error surfaces. (a) " Idle triangle". (b) " Reference triangle". Chapter 5

Conclusions

5.1 Discussion

The algorithm has proved to be an accurate tool for registration of multimodd and unirnodal three-dimensional data sets. Its performance is good enough for many applications where 3-D registration is required. The maximum error for the SPECT- teSPECT registration does not exceed the resolution of the SPECT scanner (10 mm). MR images have much better resolution than SPECT images. Hence, the orientations of the reference system can be found more accurately with the final result of better registration. However, the maximum error for the MR-teMR registration is relatively large in comparison with the resolution of MRI machine. Still, this error does not preclude using of the algorithm, since the biggest erron occured in lateral zones far beyond the region of interest. In the region of interest the errors did not exceed 2 mm, which is only 2.5 times larger than the resolution of the MRI scanner. Changes in the structure of the brain that may be overlooked due to a 2 mm registration error are not considered pat hological. Regist rat ion errors for a test MR-t O-MR registrat ion did not exceed 1 millimeter within the space of the whole data set. Good results obtained for the MR-to-MR registration prove that the algorithm con ahbe successfully used for the SPECT-to-MR registration. The algorithm uses physical constraints and appropriate assumptions. How- ever, t here are several flaws that can affect the performance of the algorithm. These drawbacks and the way to overcome them will be discussed in this chapter. 5.1.1 Reference Syst em Design

Marginal markers proved to be useful only in MR images. Resolution of SPECT images is too low to clearly discern the marker spots frorn the reference spots. Markers therefore, rnay interfere with the main tubing leading to the formation of distorted reference spots. The coordinates of the centroids of reference spots may therefore be computed erroneously. As a result, this can lead to incorrect registration. The way to deal with this problern is to use non-connected tube sections for the sides of triangles. This will elirninate the problem at the cost of convenience and additionai t ime needed to fill more tube sections. The second drawback in the reference system design is that i t does not permit saving and restoring settings, and so it cm be used on one patient only during a given study series. To provide multi-patient versatility it is necessary to restore seven sett ings of the adjustable elements. This can be done by adding graphic scales to the adjustable elements of the system. The readings on these scales can be stored in the patient's file and the reference system can be restored to its position upon request. The t hird drawback is that not dl the orientations can be registered. Triangles of the reference systern lie in planes t hat are approximately parallel to the sagittal plane. With this setup it is possible to perform only transaxial and coronal studies of the brain. Nevertheless. it is feasible to add a frame perpendicular to the principal ais of the body to permit sagittal imaging. Free space inside the MR scanner and human anatomy dictate that only one triangle can be added. Its location would be on the bridge connecting the left and right parts of the system. As the algorithm was designed to seek reference points in the left and right halves of an image, a modification to the algorithm has to be made that rotates the whole set 90 degrees. The algorithm is based on the assumption that the angle between the data sets does not exceed 30 degrees. This makes is impossible to register nearly-orthogonal data sets wi th the current configuration of the reference system.

5.1.2 Interpretation of Error Measurements

Two regist rat ion errors were measured. T hey are:

0 Errors in the locations of reference spots corresponding to the " reference triangle" , and

0 Errors in the locations of reference spots corresponding to the "idle trianglen. Observing the error surfaces in Fig. 4.2, Fig. 1.4, and Fig. 4.6, trends in the error behaviour can be estimated. Smail differences in error values for the "idle triangle" and the " reference triangle" proves t hat the regist rat ion is perforrned wi t h the same accuracy on the whole data set. It can also be seen that the registration error is more dependent on the distance from the origin of rotation than on the angle of rotation. This shows t hat once the correct determination of the centroids of reference points is achieved the correct registration is possible within the range of angles that is limited by the allowances of the equipment. The best results were achieved for MR-to-MR registration. This is justifiable, because MR images have much better resolution thao SPECT images. Hence, the orientations of the reference system can be found more accurately wit h the final result of bet ter registrat ion. Registration errors for the test MR-tc~MR regist ration did not exceed 1 millimeter within the space of the whole data set. Registration errors can also be diminished if perfectly cubic voxels were used in both data sets. Voxels in the data sets used in this thesis were cubic only in SPECT sets. In MR sets, voxels were prismatic. This voxel shape produces significant anisotropic partial volume effects. When a feature with high spatial frequency, for example, a sharp edge, crosses a physical voxel during scanning, the voxe17s physical volume is only partially "filled". However, if the grey level intensity of the feature is sufficiently high, the quantized voxel will bear this high intensity even if only small fraction of the physical voxel is "filled". This can result in incorrect measurements. With cubic voxels the partial volume effect would have less influence on registration precision.

5.2 Possible Applications

The designed algorithm is not limited in its applications with SPECT-to-MR registration. It can be used with any other imaging modality. Since SPECT is known as a method with one of the worst resolutions, the registration results cannot be worse than those obtained in this research for SPECT-to-SPECT registration. It is obvious t hat the higher the resolution, the more accurate the registration is. Fusion of the registered data was not among the problems addressed by this thesis. To visualize the results of fusion a simple technique can be suggested. Two images belonging to registered data sets are projected on the same position on the screen alternatingly. One of the images can be "moved" by the means of a "mousen about the original position by a short distance. Observing this scene while "moving" the image, a user can trace the correspondence between the features of two images. For computer analysis, images from the registered data sets can be treated separately, since t hey have sarne orientation, position, and resolut ion. The applications of the algorithm may include

O intra-patient multi-modal registration for structural and functional analysis of the brain, and

O int ra-pat ient sarne-modality regist ration for comparative observation during study or treatment periods.

5.3 Future Development

Future development of the idgorithm and method includes:

0 improvements to the reference system

- addition of a triangle in the sagittal plane, - improvement to the marginal marker design? and - development of the mechanism allowing restoration of settings.

O more experiments to estimate dependence of the registration error on orientation of the axis of rotation, and

optimization of the algorithm to reduce the computation time. References

[l] P. J. Besl and N. D. McKay, "A Method for Registration of 3-D Shapes,? IEEE Trans Pattern halys. ibiach. Intell., Vol. 14, No. 2, pp. 239-356, February 1994.

[2] L. Brand, Vector and Tensor Analysis, John Wiley md Sons, NY, 1948 pp. 403- 421.

[3] T. Turkington, R.J. Jaszczak, K.L. Greer, R. E. Coleman. and C.A. Pelizzari, "Correlation of SPECT Images of a Three-Dimensional Brain Phantom Using a Surface Fitting Technique," IEEE Trans Nuclear Science, Vol. 39: No. 5, pp. 1460-1463, October 1992.

[4] W.R. Fright and AD.Linney, "Registration of 3-D Head Surfaces Using Multiple Landmarks," IEEE Trans Med. Imaging, Vol. 12, No. 3, pp. 515-521, 1993.

(51 D. Goryn and S. Hein, "On the Estimation of Rigid Body Rotation From Noisy Data," IEEE Trans Pattern Analys. Mach. Intell., Vol. lTONo. 12, pp. 1219- 1230, December 1995.

[6] D. Boulfelfel, R.M. Rangayan, L.J. Hann, and R. Kloiber, " Prereconstruction Restoration of Myocardial Single Photon Emission Computed Tomography Im- ages," IEEE Trans Trans Med. Imaging, Vol. 11, No. 3, pp. 336-399, September 1992.

[ï] Y. Hel-Or and M. Werman, "Pose Estimation by Fusing Noisy Data of Different Dimensions," IEEE Trans Pattern Analys. Mach. Intell., Vol. 17, No. 2, pp. 195- 201, Febmary 1995.

[SI R. J. Holt and A. N. Netravali, "Uniqueness of Solutions to Three Perspective Views of Four Points," IEEE Tmns Pattern Analys. Mach. Intell., Vol. 17, No. 3, pp. 303-307, March 1995. [9] R. J. Moorhead II and 2. Zhu, "Signal Processing Aspects of Scientific Visuôliza- tion," IEEE Signal Processing Magazine, Vol. 12, No. 5, pp. 20-41, September 1995.

[IO] W. .4 . Kalender. "Quo Vadis CT? CT in the Year 2000," Siemens Electro- Medica, Vol. 61, No. 3, pp. 30-43, Febmary 1993.

[Il] K. Kanatani, " Analysis of 3-D Rotation Fitting," IEEE Trans Pattern Analys. Mach. Intell., Vol. 16, No. 5, pp. 543-550, May 1994.

[12] W.Y. Kim and A.C. Kak, "3-D Object Recognition Using Bipariate Matching Ernbedded in Discrete Relaxation7" IEEE Trans Pattern Analys. Mach. Intell.. Vol. 13, No. 3. pp. '221-251, March 1991.

[13] N. Kiryati and A. M. Bruckstein, "What is a Set of Points." IEEE Trans Pattern Analys. Mach. Intell., Vol. 14, No. 4, pp. 496-500, April 1992.

[14] 0. Kubler and G. Gerig, " Cornputer-Anaiysis of 3-D Medical Images," Siemens Reuiew, pages 4-8, Spring 1991.

[15] G.I.;. Matsopoulos, S. Marshall, and J.N.H. Brunt, "Multiresolution Morpholog- ical Fusion of MR and CT Images of the Human Brain," IEEE froc.-Vis. Image Signal Process, Vol. 141, No. 3? pp. 13'7-133, 1994.

[16] M. 1. Miller and C. S. Butler, "3-D Maximum A Posteriori Estimation for Sin- gle Photon Ernission Cornputed Tomography on Massively-Parallel Cornputers," IEEE Trans bled. Irnaging, Vol. 12 No. 3, pp. 560-566, September 1993.

[17] R.S. Mullic and N.F. Ezquerra, " Automat ic Determination of LV Orientatiton from the SPECT Data," IEEE Trans Med. Imaging, Vol. 11, No. 1, pp. 85-100, March 1995.

[18] S.P. Raya and J .K. Udupa, " Shape-based Interpolation of Multidimesiond Ob- jects," IEEE Trans Med. Irnaging, Vol. 9, No. 1, pp. 32-42, March 1990.

[19] R. A. Robb and C. Barillot, "Interactive Display and Anaiysis of 3-D Medical Images," IEEE Trans Med. Imaging, Vol. 8, No. 3, pp. 217426, September 1989.

[20] T.Shaw, Vector Calculus with Applications to Physics, D.Van Nostrad Company, NY, 1922 pp. 95-126. [21] M. Singh "Towards Proton MR Spectroscopie Imaging of Stimulated Brain Function," IEEE Trans Nvclear Science, Vol. 39, No. 4, pp. 1161-1164, August 1992.

[22] J. K. Riek, AM. Tekaip, W.E. Smith, and E. Kwok. 'Out-of-Plane Motion Compensation in Multislice Spin-Echo MRI," IEEE Trans Med. Imaging, Vol. 14. No. 3, pp. 464-471, September 1995.

[23] S. D. Stearns and R. A. David. Signal Processing Algorithms. Prentice-Hall. 1988 pp. 276-277.

[24] 1. Carlbom, D. Ter~opoulos~and K.M. Harris. 'cornputer- Assisted Registrat ion, Segmentation and 3D Reconstruction from Images of Neuronal Tissue Sections ," IEEE Trans Med. Imaging, Vol. 13, No. 2, pp. 351-363, February 1994.

[25] T.Herbert and R. Leahy, " A Generalized EM Algorit hm for bD Bayesian Recon- struction from Poisson Data Using Gibbs Priors," IEEE Trans Med. Imaging, Vol. 8, No. 2, pp. 194-202, June 1989.

[26] S. Umeyama, " Least-Squares Estimation of Transformation Parameters Between Two Point Patterns," IEEE Trans Pattern Analys. Mach. Intell., Vol. 13, No. 4, pp. 376-350, April 1991.

[27] C. M. Meyer. G. H. Leichtman, J. A. Brunberg, R.L. Wl.and L.E. Quint, "Simultaneous Usage of Homologous Points, Lines. and Planes for Optimal, 3- D. Linear Registration of Multumodali ty Imaging Data," IEEE Trans Med. Imaging, Vol. 11, No. 1, pp. 1-11, March 1995.

[28] M. Werman and D. Weinshall, "Similarity and Affine Invariant Distances Be- tween 2D Point Sets." IEEE Trans Pattern Analys. Mach. Intell.. Vol. 17' No. 8, pp. 810-814, Aug 1995.

[29] R. Sauter, M. Shneider, K. Wicklow, and H. Kolem, "Current Status of Clinically Relevant Techniques in Magnet ic Resonance Spec t roscopy," Siemens Electro- Medica, Vol. 60, 29-56, pp. 32-54, February 1992.

[30] A.Venot, J.Y.Devaux, M.Herbin, J.F.Lebruchec L.Dubertret, Y-Raulo, and J .C. Roucayrol, " An Automated System for Registration and Comparison of Photographie Images in Medicine," IREE Trans Med. Imaging, Vol. 7, No. 4, pp. 304-312, December 1988. Appendix A

Quaternion Mat hematics

The algebra of quaternions was introduced by Sir Williaxn Rowan Hamilton in 1543 as a solution to the problem of extending bdimensional vector algebra to include multiplication and division. Quaternions include the real numbers (xl0,0, O) with a single unit 1, and the complex numbers (x, y, O: O) with two units 1. i. Both real and complex numbers form a field, that is a set of numbers in which the sum, difference, product and quotent (the divisor not being zero) of two numbers of the set must belong to the set. Quaternions include vectors (O, x. y, r ) in a space of t hree dimensions. Because quaternions include vectors they are suitable for expressing the problem under consideration. With the aid of quaternion algebra, finite rotations in space may be dealt with in a simple and elegant manner. The following theorern is given in [lz]:

Theorem 3 Any quaternion q can be written in the folloving form:

,where

77- z, 3, k = triple de f ining the Cartesian coordinate system, dl a,b, c = real coefficients, S, = d is the scalar part of the quaternion q and V, = aT+ 67 + CG is the uedor part of the quaternion q.

A.1 Conjugate and Norm of Quaternions

The conjugate of a quaternion q, written h;, is defined as

Kq = Sq - 4

The norm of a quaternion q, written iV, is a scalar defined as

!Vq = q(&) = d2 +a2 +b2 + c2 (A-3)

If A$ = 1, q is called a unit quaternion. If 1% = O, q is called a zero quaternion.

A.2 Product of Quaternions

The quaternion product of two quaternions q and q1

is obtained by distributing the terms on the right as in ordinary aigebra, except that the order of the units must be preserved, and then replacing each product of units by the quantity given in the following multiplication table:

Table A. 1: Quaternion multiplication table. A.3 Division of Quaternions

If q is not zero, Nq is a non-zero scalar: and the defining equation for the norm may be written as

Therefore 3 can be defined as the reciprocal of q.

(Am

Division of quaternions can now be performed by multiplication by a reciprocai quaternion. Every quaternion

q=d+a;+b;+ck (A.7) wit h real coefficients may be mitten as a real multiple of a unit quaternion:

q = h(cos Q + ësin Q),O < Q < 2n w here

And, when a* + b2 + cZ # 0, ë is the unit vector: Appendix B

Program Listings

B.1 First Module

The purpose of the First Module is to analyze a set of images. The coordinates of the valid reference points in valid slices are stored in two files, for the left and right halves of the analyzed images.

/@ proaptr rooc nur. sitoaiu tomAc: zs ts mnu: holpttlo: protocypo: tnc chanf. (chu *roocauo. chu *#avenan ; @/

tnc chowmk (chu *rootnuo. chu *savon.u) camp - .t e mxpu(l.0); /a 0.05 for phmto=. 0.1 for brain 0.3 for rpct */ bînuy(1.2. Camp) : displiy<2. 2. 'r");

ciurrhcomp(2. 3. 2. 35); /O (2.35) for phint08 and briin (5.50) for rpcc a/ wk(O.3.41; diSpliJ(4. 2. -5"): camp - .3 ruph(4.0); binuy(4.3, camp); dfsphy(3. 1. "8");

/* Counc mfiranca points in aich blti of cha iuga */ rk.. rom. rnd neror roforonco pornts rn racb luli of tbr riy.. tirouig lavalid paiiic. */

ud - rangr(ri3 M ; rragrtcî3COl - rrngaWC01; rrapCR3C03 wd; wd = ringocri3 Cd; rrapLrl3 Ci3 = range W Cl1 ; rango W Cl3 - rd: rd riagak1lW: r.npWW - rinpCtz3Cü; rraptr2lW rd: ? Li (r.ngrCrdM - O II rrngotr2lC23 - Oli rang Cri3 Col - r-ga W Ctl rangeGa C23 - rwpWDl - 0: rangrWC01 - ranpWCfl rriyrWN - r.ryWD1 O; /* 1p1tt Spitc on* il10 in ma for ch. Sacoad Podulo uiput */

; */

Sincludo "crcd .h" tiacludo cstdto .h> Sincludo Sinclildo

me aplir (chu

B.2 Second Module

The Second Module is programmed to read the narnes of the files specified in the command line and to produce a new "registered" set as a result.

/* prompt: tirsr rooc mur. firrc rtrr nur. sacond rooc nw. socond star na. ris nam format: xs Zr xs Xl xs xs mana : hrlpiil.: prototypo: int readup(chu wfrootnii.. chu itrnan. chu ~iroorn.u.chu aisaur. chu *ripan); */

readop (chu afroocnmo. chu *iraamo. chu ~rrootnau.chu *rsn.u. chu arrpu.)

I PILE in0 iat tlort chu chu ch Chu chu t1o.t tloit iapointrC2J[ iilopointor - fapn(ssnru. "rl ; if(filopoiator - IUU)i printfC" rwblo ta opan socond\s"): 0xitCO): ) fscrof (iil*polator. "Id" td U U UV'.RSXSIZE. MTSIZt. -1. WUâL. -1) ;

int i. j. k. 1. m. p. x. y. x. xi. ys. 1s. rndox: int Lay. koyt. ily. pmCS3; lon6 tnt jtinllongth. offroc; short pkrl. plxoll. tamp. pixxo. pixyo. pw. php. LottpL.. rbhcpb. lorpix. uppu;

iloit 1r.r. Ir*.. Ifri. 1-1. cres. crem. etri. cfrr: iloat mer. ptr. zaocr. coralpha. corkrr. t.kdotfri. r01-dot-trs; flair qCU. qltU. +CU; ilaar frs CU. -CU .trnf#. romCU . rasCU. trrcfrCU . tostroCU ; tlort 4i.c. distiin. lonpoct. but; ilarc rbUtCU. i~ortico~~.rwrtic*~u: ilorr sreccorCU. sr+*ctarCU ; iloac u. yr. tr. ibiu. ybiu. zbtu. xbir. ybla. xbia. rquur; flaat spui. ipl.rpuux. lm. rpuUJ: sprnrx - tinpo tacs CllCO1 CO1 - tinpa Lncs (01 CO1 COI ; spuq - tiapo hcsCiIC01 Cl1 - tinpoincsC03 COI Cl3 ;

squuo - (tinpoincsc0lclJco3 (tinp. intsCO3 COI COI (tinpoints CO3 ici COI (f inpointa COI COI COI

nmrticaCO1 - 0; nonic.Ct1 - (

craa - lonpact/lroa: cris - loapact/lras: ch loagvact/lm; cira - loagact/lirs ; CU. O: filmpointer = tpO: bnak; cue 1: f ilepainter - ipl; break; cue 2: f ilepainter - fp2; brmak; cue 3: tilrpoincer - fp3; broak; CU. 4: filepointor - tpl; break; CU. 5: fuep0hCer - bmak; cue 6: f ilapointer - fp6; broak; CU* 7: f ilepoincmr - ?pl; brmak; CU. 8: iilmpointrr - tpa; break; cue 9: f ilepointrr - fp9; bmak; CUI 10: f ilepoincer - fplO; brmak; CU0 11: ?ifbpokDc*r bm&; CU. 12: filepointer - tp12; bmak; cue 13: tilepoincrr - fpl3; brmak; cue t4: iilepointer - Q14; break; cur 15: ttlepointer - tp1S; bruk: CU. 16: iilrpointer - -16; break; eue 17: tilepoiacer - Q17: bnak; cur 18: tilepoiatir - fp16: break; cue 19: f ilepeinter - -19; break; cue 20: f ilapointer - Q20; break; cua 21: iihpoiatar - -21: bnak; cur 22: iileprintrr -22; bmak: cue 73: tilrpointer fpU; brmak: cur 24: filepointer - tp24: brmak; cue 1i: f ilepotnter - tp25: break; cue 26: filepointer - fp26; brmak; cur 27: filmpolnter - ip27: bnr*; cue 28: tilrpoincrr - fpm; break; eue 29: filepainter - ip29; bnak; deiaulc: filepointer 0; break: 1

pixel 9 hftpix righcph uppir - lo.pix - 0;

/* Iacrrpolat ion --r/

/* - Assaring ChrC point ts ta one a? ch. 6 quadrants -*/ /* -fictif16 krys drponding 01) Che qaadruàc (for furCher siitchl -*/

UCxbiu CO U ybtu CO i Irryt-5; iftxbiu >O ) k.y ' 5; alre key - 6; ?

CU* 1: piuo - righcpîx: puy0 - uppL: brrak;

CU. 3: ph0 ' leitptr; piryo - upptr; break; eu* 5: p)no - lritptr: pLxy0 - lampk; brrak; cur 7: pfuo - rfghcpk; ptxyo - lompix; break ;

cur O: tîlrpriarer fpO; bnak; car 1: tilrprtaCrr - ipl; bnrl; cur 2: tilrpointrr - fp2; bmak: cur 1: tilrpaiatrr fp3; brrak; cur 4: tilrpriatrr fp4; brmk; cur 5: f ilrpoiacrr tp5; break; CU. 6: filrpointrr fpo; broak; cur 7: tilrpoiatrr fp7: break; CUI 8: fil.p~ht*r ' fpd; bmak; cur 9: f ilrpointrr * -9: broak; cur 10: iil~poiocrr9 fp10; broak: cur il: filrpoiotrr fpll; broak; CU* 12: tilrpointrr tpl2; break: cur 13: tilrpointrr +if: brrak; cur 14: tilrpoiotrr - fpl4: break: cur 15: filrpofntrr 9 qi5; broak; cur 16: filepointrr fp16; brorir; CU* 17: tilrpointrr - tp17: brrak: cur 18: iilrpointrr fp18; bnak; cur 19: filepointrr - tp19; brrak; cur 20: f ilrpofntrr - tplO: broak; sasr 21: tilrpointrr tp21; break; cur 22: tilrpointrr - fp22; break; cur 23: iilrpoincrr tpî3; break; cur 24: iihpointrr - fp24; bmak: cur 15: tilrprintrr tp25; brmak; CU* 26: tilrpofntrr - fp26: bnak; cur 27: tilrpointrr - tp17: brrak: cur 28: iilrpointrr tp28; brrrlt; CU* 2929: tflrpointrr - -29: bnak; drtiulc: tilrprintrr O; brrak; >

if(fi1rpoincrr -9 ICN II (2s - 1) c O II (zs-1) >- StSXZe) phrll-httph - rightpix ' ~pph 10.ph piI.1; rlrr C ti(0rfi.t > 4 I I off1.t < S~IZL.(StSIZE-2)4) c trrrk(ti1rpolntrr. riziat(oasiprd cbu)aottrrt. 0); trrid(l1oupix. rizrotCunsipid shorr). 1. tilrpointerl ; lo.pt*(shorci loipi. :

trr.d(Rboiirr. sluot(uariprd chu). 5ISIZëa2-4. tilrpointrr); trr.d(Ruppix. siuot(unsigrd short). 1, tilrpoincrr) ; upph=

cur 0: filrpointrr - fpO: brrak; cur 1: fflrpbincrr tpl; brrak; cur 2: iilrpointrr - tp2; break: CU* 3: tilrpoincrr - tp3; brmak: cur 4: tilrpoincrr - fp4; broak; cur 5: tilrpofatrr - ip5; brmak; CU* 6: ttlrprincrr fpb; break; cur 7: tilrpoincrr - tp7; briak; eue 8: ftlrpointrr - te: brrak; cur 9: tilrpsintrr - tp9; brrak; cur 10: tilrpointrr ip1O; break; eue 11: tilrpofatrr fpli; break; cur 12: filrpofntrr - tpl2; broc; CU. 13: tilrpoinC*r - tp13; break; sur 14: tilrpointer - Q14; brrak; cur 16: tflrpointrr ip1S; brrak: cur 16: filrpotatrr ip16; brmak; cur 17: tilrpointrr tplf; break; cur 18: tilrpointrr - fpl8: brrak: cur 19: tilrpriatrr fpt9; brrak; cur 20: tilrpointrr Q2û; brrak; cur 21: tilrpoiatrr -21: brmak: cur 22: tilrpefacrr fp22; break; cur 23: filrpoiatrr fp23: bnak; cur 24: tilrpbintrr fp24; break; cur 15: tilrprlncrr fpS; brmak; U(ti1.point.r - nRL 11 (u*t) .-SZSIZL I I (ts+11<0) pixall-lottpix rightpix = uppir loipix pixel; rlso i

frod(kbiifi.r. s~or(pnsipodchu). SEXZE*~-~. fiIepoincer) ; tnd(&ldtpix. siuoi(unsiped short). 1. illepointer); loirpit-(short) loitpix:

CU. 1: piur - rightpix; pixyz ' upptr: brmak:

CU. 3: piru ldtptr; phyz - uppix; break; CU0 5: pFxu 9 leitpix: pixyz ' lorpir; OrOak : CU0 7: p-r - rightpix: pirp loipix: break: tloac ql. lmogprod; f1o.t slabAIi; bt i; tloat *CU;

"qv" /*- iunction pariaru rotation q al 4 i*ctor 1 inco iactor v-a char is dalined by ch* qurtrrnron q --*/

/* - Fiinction "99" m~ltipliewcio quataraions /

/*-Cuacttan 'quat2" crlculat~rth* qmicernioo ciuc conlcrlly rrCat*s rocter v2 hco v*ctor 13 .bout 11 */

IMAGE EVALUATION TEST TARGET (QA-3)