DOCSLIB.ORG

Software Approach to Merging Molecular with Anatomic Information

Total Page：16

File Type：pdf, Size：1020Kb

Download full-text PDF Read full-text
Software Approach to Merging Molecular with Anatomic Information Software Approach to Merging Molecular with Anatomic Information Piotr J. Slomka, PhD Departments of Imaging and Medicine, Cedars-Sinai Medical Center and UCLA School of Medicine, Los Angeles, California; and Department of Diagnostic Radiology and Nuclear Medicine, University of Western Ontario, London, Ontario, Canada ized and are used clinically in several centers. In this article, Software image registration is a powerful and versatile tool that issues related to the practical implementation of software allows the fusion of molecular and anatomic information. Image for merging anatomic and functional information are dis- registration can be applied to compare anatomic information cussed, including: (a) description of computer algorithms with function, localize organs and lesions, and plan radiation for automatic retrospective image registration; (b) valida- therapy, biopsy, or surgery. Automatic volume-based image tion of accuracy for such algorithms; (c) visualization tech- registration techniques have been devised for both linear and nonlinear image alignment. Challenges remain in the validation niques for display of fused images; (d) clinical applications; of the accuracy of software registration. Image registration has and (e) comparison with hardware PET/CT technology. been applied clinically in neurology and oncology and may be particularly practical in radiotherapy applications. Potential new applications in cardiology could allow the combination of CT IMAGE REGISTRATION ALGORITHMS angiography with perfusion and viability images obtained by Several approaches have been proposed for the retrospec- PET, SPECT, or MRI. Software methods allow versatility in the tive automatic registration of multimodal images. Broadly, choice of modalities and facilitate retrospective and selective these registration algorithms could be grouped as feature application. Fully automatic registration algorithms are needed for routine clinical applications. Connectivity, compatibility, and based (using extracted image features) and volume based cooperation between various clinical departments are essential (using statistical voxel dependencies). The algorithms also for the successful application of software-based image fusion in could be classified as linear, when computed alignment a hospital setting. transformation between 3D image volumes is limited to Key Words: image registration; image fusion; PET/CT; nonlin- translation, rotation, and possibly scaling, or nonlinear, ear registration; mutual information which allow more complex transformations. Nonlinear J Nucl Med 2004; 45:36S–45S techniques can be feature based or volume based. Feature-Based Algorithms Feature-based registration algorithms seek to align cor- responding anatomic landmarks, organ surfaces, or other usion of images containing molecular and anatomic F features. Such techniques consist of 2 steps: (a) extraction of information could aid clinicians in a variety of clinical relevant features (points, contours, surfaces) from the im- applications, including comparison of anatomic information ages; and (b) spatial alignment of these features. Two rep- with function, localization and boundary definition of or- resentative examples of the feature-based approach are the gans and lesions, planning of radiation therapy and biopsy, “head and hat” method (1) and the “iterative closest point” and integration of PET or other functional modalities with method (2). Accurate image segmentation is required be- image-guided surgery. cause the registration relies on only the extracted features. Merging of multimodality images requires accurate im- Therefore, errors in image segmentation will inevitably lead age alignment, which is typically referred to as image reg- to errors in image registration. Although the registration of istration. Such image registration can be accomplished by accurately segmented surfaces is computationally straight- software algorithms that retrospectively align 3-dimensional forward, the identification of such surfaces may be difficult (3D) data acquired by stand-alone modalities to common and prone to errors. This step may require significant user spatial coordinates. Practical systems using software regis- interaction, even in the case of brain registration. Automatic tration algorithms for image fusion have been commercial- extraction of features needs to be customized for each imaging modality and for each organ of interest. Some Received Sep. 18, 2003; revision accepted Nov. 7, 2003. registration techniques rely on external fiducial markers that For correspondence or reprints contact: Piotr J. Slomka, PhD, Department can be identified and matched (automatically or manually) of Imaging, Cedars-Sinai Medical Center, A047 8700 Beverly Blvd., Los Angeles, CA 90048. on the acquired images (3,4). The main limitations of such E-mail: [email protected] marker-based approaches are the increased complexity of 36S THE JOURNAL OF NUCLEAR MEDICINE • Vol. 45 • No. 1 (Suppl) • January 2004 the imaging procedures and the lack of information about reliable than the registration of emission or transmission internal organ displacements. data alone (11). It is also possible to adjust the image resolution “on the fly” during the iterative search for the Volume-Based Algorithms best alignment. Initially, the stand-alone images can be More recently, volume-based image registration tech- grossly misaligned, and small matrix sizes are sufficient to niques have been introduced to maximize measures of sim- search for approximate alignment. The matrix size can be ilarity (cost function) between images. The proposed mea- progressively increased as the images become closely sures of the alignment quality include the standard deviation aligned, thus allowing fine adjustments. Such multiresolu- of the histogram (5), joint entropy (6), and mutual informa- tion techniques can decrease the time of the computations tion (MI) (7,8). In particular, methods implementing MI and may avoid entrapment in “local minima” (15). A measures (Fig. 1) have been proven versatile and successful method of search for the optimal value of the cost function in clinical applications (9). Volume-based techniques usu- may affect the calculation time or entrapments in local ally do not depend on image segmentation but exploit the minima. Furthermore, the number of registration parameters statistical voxel dependencies of the raw image pairs to find will determine the time required to find a solution. For the appropriate alignment. These techniques were initially example, the search for a transformation that includes ad- designed for the registration of MRI, CT, and PET images ditional scaling parameters will take longer than rigid-body of the brain (5,7,8) but recently have been extended to other registration. Current practical implementations allow fully organs (10–12). Volume-based techniques have been automated image registration of thoracic CT and PET in Ͻ1 shown to achieve better accuracy than surface-based meth- min (10,11) with accuracy at Ͻ1 cm. ods (13,14). Several possible modifications to volume-based algo- Nonlinear Registration rithms can improve their reliability and speed. For example, Image registration of thoracic and abdominal scans may PET transmission maps acquired with emission data can be require nonlinear transformation to compensate for changes used in the calculation of the cost function, because these in body configuration, breathing patterns, or movements of maps do not reflect the physiologic variations of the radio- internal organs. Nonlinear image alignment (image warp- isotope uptake. The emission images, however, contain ing) uses advanced interpolation schemes, such as the thin- important information that can correlate with anatomic fea- plate spline method (16) or piecewise-linear methods (12) tures on CT or MRI. Therefore, the registration of the adapted to 3 dimensions. A major difficulty with nonlinear combined emission and transmission data with CT is more warping is the determination of the correct nonlinear trans- FIGURE 1. Concept of image registration based on mutual information (MI) is explained using example of PET and CT. Separate PET and CT image intensity histograms are derived from PET and CT, which contain frequencies (f) of occurrence for specific voxel values in 3D volumes (p ϭ PET, c ϭ CT). Additional 2D image histogram is created from combination of PET and CT data, in which frequencies of occurrence for particular PET/CT voxel intensity pairs (p, c), both at same location, are calculated. Subsequently, PET and CT image entropies are calculated from PET and CT histograms, and 2D PET/CT histogram is used to calculate joint entropy. Joint entropy is smallest and, consequently, MI is largest when images are closely aligned and 2D histogram is least dispersed. Search is performed, which continuously modifies 3D shifts (X,Y,Z) and rotations (XY, XZ, YZ), each time transforming PET data. Although it is possible to perform image registration using joint entropy only, inclusion of separate PET and CT entropies is needed when portions of PET volume could move outside of overlapping field of view. SOFTWARE REGISTRATION AND FUSION • Slomka 37S form from the functional/molecular images. The transmis- the algorithms (18). Moreover, no available phantoms sim- sion images are acquired during the same tidal breathing as ulate complex patient motion. The accuracy of the volume- the emission images and, therefore, may provide an approx- based PET-to-CT image registration for the brain has been
Load more

Recommended publications
  • Image Fusion for Conformal Radiation Therapy
    IMAGE FUSION FOR CONFORMAL RADIATION THERAPY Marc L. Kessler, Ph.D. and Kelvin Li, M.S. Department of Radiation Oncology, The University of Michigan 1. INTRODUCTION 2. IMAGING MODALITIES IN TREATMENT PLANNING A. X-ray CT B. Magnetic Resonance Imaging C. Nuclear Medicine Imaging D. Ultrasound Imaging 3. TECHNIQUES A. Dataset Registration 1) Surface-based Registration 2) Image-based Registration 3) Interactive Registration B. Data Mapping and Image Fusion 1) Structure Mapping 2) Image Mapping and Fusion 4. CLINICAL EXAMPLE 5. SUMMARY 6. REFERENCES Acknowledgements This paper is reprinted with permission from 3D Conformal Radiation Therapy and Intensity Modulated Radiation Therapy: Physics and Clinical Considerations. Ed. J. A. Purdy, W. Grant III, J. R. Palta, E. B. Butler, C. A. Perez, Madison, WI, Advanced Medical Publishing; 2001. 1. INTRODUCTION Medical imaging is now a fundamental tool in conformal radiation therapy. Almost every aspect of patient management involves some form of two or three dimensional image data acquired using one or more modality. Image data are now used for diagnosis and staging, for treatment planning and delivery and for monitoring patients after therapy. While x-ray computed tomography (CT) is the primary modality for most image-based treatment planning, other modalities such as magnetic resonance imaging (MR), positron and single photon emission computed tomography (PET and SPECT) and ultrasound imaging (US) can provide important data that may improve overall patient management. For example, MR provides excellent soft tissue contrast, allowing superior delineation of normal tissues and tumor volumes in many sites. SPECT and PET provide unique metabolic information capable of resolving ambiguities in anatomic image data and can quantify partial organ function.
    [Show full text]
  • Ultrasound and Magnetic Resonance Image Fusion Using a Patch-Wise
    Ultrasound and magnetic resonance image fusion using a patch-wise polynomial model Oumaima El Mansouri, Adrian Basarab, Mario Figueiredo, Denis Kouamé, Jean-Yves Tourneret To cite this version: Oumaima El Mansouri, Adrian Basarab, Mario Figueiredo, Denis Kouamé, Jean-Yves Tourneret. Ultrasound and magnetic resonance image fusion using a patch-wise polynomial model. IEEE Inter- national Conference on Image Processing (ICIP 2020), Oct 2020, Abu Dhabi, United Arab Emirates. pp.403-407, 10.1109/ICIP40778.2020.9191013. hal-02982900 HAL Id: hal-02982900 https://hal.archives-ouvertes.fr/hal-02982900 Submitted on 29 Oct 2020 HAL is a multi-disciplinary open access L’archive ouverte pluridisciplinaire HAL, est archive for the deposit and dissemination of sci- destinée au dépôt et à la diffusion de documents entific research documents, whether they are pub- scientifiques de niveau recherche, publiés ou non, lished or not. The documents may come from émanant des établissements d’enseignement et de teaching and research institutions in France or recherche français ou étrangers, des laboratoires abroad, or from public or private research centers. publics ou privés. Open Archive Toulouse Archive Ouverte OATAO is an open access repository that collects the work of Toulouse researchers and makes it freely available over the web where possible This is an author’s version published in: http://oatao.univ-toulouse.fr/26432 Official URL https://doi.org/10.1109/ICIP40778.2020.9191013 To cite this version: El Mansouri, Oumaima and Basarab, Adrian and Figueiredo, Mario and Kouamé, Denis and Tourneret, Jean-Yves Ultrasound and magnetic resonance image fusion using a patch-wise polynomial model.
    [Show full text]
  • PET/CT: Panacea, Redundancy, Or Something in Between?
    PET/CT: Panacea, Redundancy, or Something in Between? Wouter V. Vogel, MD1; Wim J.G. Oyen, MD1; Jelle O. Barentsz, MD2; Johannes H.A.M. Kaanders, MD3; and Frans H.M. Corstens, MD1 1Department of Nuclear Medicine, University Medical Center Nijmegen, Nijmegen, The Netherlands; 2Department of Radiology, University Medical Center Nijmegen, Nijmegen, The Netherlands; and 3Department of Radiotherapy, University Medical Center Nijmegen, Nijmegen, The Netherlands accuracy to many diagnostic regimens compared with ana- In the past decade, the integration of anatomic imaging and tomic imaging only (CT, MRI, ultrasound). For several rea- functional imaging has emerged as a new and promising diag- sons, anatomic and functional imaging has been integrated into nostic tool. Developments in software provided methods to one diagnostic modality that is known as image fusion. integrate various modalities, such as PET, CT, MRI, and MR Image fusion can be performed at 3 different levels: spectroscopy. The introduction of combined PET/CT scanners visual fusion, software fusion, and hardware fusion. In has boosted image fusion in this specific field and raised high expectations. Image fusion can be performed at 3 different traditional visual image fusion, the physician compares 2 levels: visual fusion, software fusion, and hardware fusion, each separate imaging modalities viewed next to each other. The having strengths, weaknesses, and issues inherent to tech- fusion takes place in his or her mind. In soft- and hardware nique. Visual fusion is the traditional side-by-side reviewing of 2 image fusion, the results of both procedures are overlaid in separate modalities. Software image fusion provides evaluation an integrated set of images.
    [Show full text]
  • Ct-Pet Image Fusion and Pet Image Segmentation for Radiation Therapy
    CT-PET IMAGE FUSION AND PET IMAGE SEGMENTATION FOR RADIATION THERAPY by Yiran Zheng Submitted in partial fulfillment of the requirements For the degree of Doctor of Philosophy Dissertation Adviser: Barry W. Wessels, Ph.D. Department of Biomedical Engineering CASE WESTERN RESERVE UNIVERSITY January, 2011 CASE WESTERN RESERVE UNIVERSITY SCHOOL OF GRADUATE STUDIES We hereby approve the thesis/dissertation of Yiran Zheng candidate for the Ph.D. degree *. (signed) Andrew M. Rollins, Ph.D. (chair of the committee) Xin Yu, Ph.D. Barry W. Wessels, Ph.D. Syed F. Akber, Ph.D. ________________________________________________ ________________________________________________ (date) July 12th, 2010 *We also certify that written approval has been obtained for any proprietary material contained therein. Dedication To my parents and my wife Table of Contents Table of Contents ............................................................................................... 1 List of Tables ...................................................................................................... 4 List of Figures .................................................................................................... 6 Acknowledgements ............................................................................................ 8 List of Abbreviations ........................................................................................ 10 Abstract ............................................................................................................ 12 Chapter
    [Show full text]
  • CT-MRI Image Fusion Effect on Dose Delivered and Distribution for Astrocytoma Radiation Therapy
    Central Journal of Radiology & Radiation therapy Research Article *Corresponding author Mariam Isam, B.Sc Biophysics, Faculty of Science, Mansoura University, Egypt, Tel: +201022882702; CT-MRI Image Fusion Effect on Email: Submitted: 14 November 2014 Dose Delivered and Distribution Accepted: 01 June 2015 Published: 05 June 2015 ISSN: 2333-7095 for Astrocytoma Radiation Copyright © 2015 Isam et al. Therapy OPEN ACCESS 1 2 1 Tamer Dawod , EM Abdelrazek , Hend Ahmed El-Hadaad and Keywords Mariam Isam3* • Astrocytoma 1Department of Clinical Oncology and Nuclear Medicine, Mansoura University, Egypt • CT 2Department of Physics, Mansoura University, Egypt • MRI 3Department of Science, Mansoura University, Egypt • Image fusion Abstract Aim: This study described the benefits of CT-MRI fusion to consolidate the accuracy of conformal radiation therapy for Egyptian astrocytoma patients. Materials and methods: Thirty four patients with astrocytoma tumor who underwent 3D conformal radiotherapy were included in this study. CT and MRI imaging was done for all the patients after surgery. The PTVCT and PTVMRI tumor were measured and compared with each other. The center of mass (CM) of the PTV delineated on CT and MRI were computed and the shift between the two CMs was determined. Results and Discussion: The mean and median for PTV measured by CT-MRI was 66.77cm3±48.77 and 43.65 (range 8.6 to 162.2) respectively. For CT the mean and median for PTV was 53.82cm3± 45.66 and 31.7 (range 12.4 to 180.1) respectively. The differences between the centers of mass (CM) of the two PTV’s detected spatial 3D shifts ranging from (-4mm to 5mm) with an average of 0.3 mm for x direction, (-2mm to 5mm) with an average of 1.3 mm for y, and (-2mm to 4mm) with an average of 0.7 mm for z direction respectively.
    [Show full text]
  • Medical Image Fusion: a Survey of the State of the Art, Information Fusion, 2014
    Cite as: A.P. James, B. V. Dasarathy, Medical Image Fusion: A survey of the state of the art, Information Fusion, 2014 Medical Image Fusion: A survey of the state of the art A. P. Jamesa, B. V. Dasarathyb a Nazarbayev University, Email: [email protected] b Information Fusion Consultant Abstract Medical image fusion is the process of registering and combining multiple images from single or multiple imaging modalities to improve the imaging quality and reduce randomness and redundancy in order to increase the clinical applicability of medical images for diagnosis and assessment of medical problems. Multi-modal medical image fusion algorithms and devices have shown notable achievements in improving clinical accuracy of decisions based on medical images. This review article provides a factual listing of methods and summarizes the broad scientific challenges faced in the field of medical image fusion. We characterize the medical image fusion research based on (1) the widely used image fusion methods, (2) imaging modalities, and (3) imaging of organs that are under study. This review concludes that even though there exists several open ended technological and scientific challenges, the fusion of medical images has proved to be useful for advancing the clinical reliability of using medical imaging for medical diagnostics and analysis, and is a scientific discipline that has the potential to significantly grow in the coming years. Keywords: Image Fusion, Medical Imaging, Medical Image Analysis, Diagnostics 1. Introduction Medical image fusion encompasses a broad range of techniques from image fusion and general information fusion to address medical issues reflected through images of human body, organs, and cells.
    [Show full text]
  • Application of Image Fusion in Diagnosis and Treatment of Liver Cancer
    applied sciences Review Application of Image Fusion in Diagnosis and Treatment of Liver Cancer Chengxi Li 1,* and Andrew Zhu 2 1 Department of Mechanical Engineering, Massachusetts Institute of Technology, Cambridge, MA 02139, USA 2 Harvard Medical School, Boston, MA 02115, USA; [email protected] * Correspondence: [email protected] Received: 3 January 2020; Accepted: 5 February 2020; Published: 9 February 2020 Abstract: With the accelerated development of medical imaging equipment and techniques, image fusion technology has been effectively applied for diagnosis, biopsy and radiofrequency ablation, especially for liver tumor. Tumor treatment relying on a single medical imaging modality might face challenges, due to the deep positioning of the lesions, operation history and the specific background conditions of the liver disease. Image fusion technology has been employed to address these challenges. Using the image fusion technology, one could obtain real-time anatomical imaging superimposed by functional images showing the same plane to facilitate the diagnosis and treatments of liver tumors. This paper presents a review of the key principles of image fusion technology, its application in tumor treatments, particularly in liver tumors, and concludes with a discussion of the limitations and prospects of the image fusion technology. Keywords: medical image methods; liver cancer diagnosis and treatment; image fusion 1. Introduction Medical imaging equipment has developed rapidly in the last decade, with widespread usage in clinical diagnosis and treatment. Two main imaging modes are employed that utilize different principles and equipment. The first is the anatomical imaging mode that mainly provides anatomical information with high resolution. The X-ray-based method, computed tomography (CT), which falls into the category of anatomical imaging, is the first technique developed for the noninvasive acquisition of images within the human body.
    [Show full text]