Biomedical Imaging Informatics 9 Daniel L. Rubin , Hayit Greenspan , and James F. Brinkley After reading this chapter, you should know the assembled into a pipeline when creating imag- answers to these questions: ing applications? • What makes images a challenging type of data • What is an imaging modality with high spatial to be processed by computers when compared resolution? What is a modality that provides to non-image clinical data? functional information? Why are most imag- • Why are there many different imaging modal- ing modalities not capable of providing both? ities, and by what major two characteristics do • What is the goal in performing segmentation they differ? in image analysis? Why is there more than one • How are visual and knowledge content in segmentation method? images represented computationally? How are • What are two types of quantitative informa- these techniques similar to representation of tion in images? What are two types of seman- non-image biomedical data? tic information in images? How might this • What sort of applications can be developed to information be used in medical applications? make use of the semantic image content made • What is the difference between image regis- accessible using the Annotation and Image tration and image fusion? What are examples Markup model? of each? • What are four different types of image pro- cessing methods? Why are such methods 9.1 Introduction D. L. Rubin , MD, MS (*) Imaging plays a central role in the health care Departments of Radiology and Medicine , Stanford process. The fi eld is crucial not only to health University , 1201 Welch Road, P285 , care, but also to medical communication and Stanford 94305 , CA , USA education, as well as in research. In fact much of e-mail: [email protected] our recent progress, particularly in diagnosis, can H. Greenspan , PhD be traced to the availability of increasingly Department of Biomedical Engineering, Faculty of Engineering , Tel-Aviv University , sophisticated imaging techniques that not only Tel-Aviv 69978 , Israel show the structure of the body in incredible e-mail: [email protected] detail, but also show the function of the tissues J. F. Brinkley , MD, PhD within the body. Departments of Biological Structure, Biomedical Education and Medical Education, Computer Science and Engineering , University of Washington , 357420 , This chapter is adapted from an earlier version in the third Seattle 98195 , WA , USA edition authored by James F. Brinkley and Robert e-mail: [email protected] A. Greenes. E.H. Shortliffe, J.J. Cimino (eds.), Biomedical Informatics, 285 DOI 10.1007/978-1-4471-4474-8_9, © Springer-Verlag London 2014 286 D.L. Rubin et al. Although there are many types (or modalities) The major topics in biomedical imaging infor- of imaging equipment, the images the modalities matics include image acquisition, image content produce are nearly always acquired in or con- representation, management/storage of images, verted to digital form. The evolution of imaging image processing, and image interpretation/ from analog, fi lm-based acquisition to digital for- computer reasoning (Fig. 9.1 ). Image acquisi- mat has been driven by the necessities of cost tion is the process of generating images from reduction, effi cient throughput, and workfl ow in the modality and converting them to digital form managing and viewing an increasing prolifera- if they are not intrinsically digital. Image con- tion in the number of images produced per imag- tent representation makes the information in ing procedure (currently hundreds or even images accessible to machines for processing. thousands of images). At the same time, having Image management / storage includes methods images in digital format makes them amenable to for storing, transmitting, displaying, retriev- image processing methodologies for enhance- ing, and organizing images. Image process- ment, analysis, display, storage, and even ing comprises methods to enhance, segment, enhanced interpretation. visualize, fuse, or analyze the images. Image Because of the ubiquity of images in biomedi- interpretation / computer reasoning is the pro- cine, the increasing availability of images in digi- cess by which the individual viewing the image tal form, the rise of high-powered computer renders an impression of the medical signifi cance hardware and networks, and the commonality of of the results of imaging study, potentially aided image processing solutions, digital images have by computer methods. Chapter 20 is primarily become a core data type that must be considered concerned with information systems for image in many biomedical informatics applications. management and storage, whereas this chapter Therefore, this chapter is devoted to a basic under- concentrates on these other core topics in bio- standing of the unique aspects of images as a core medical imaging informatics. data type and the unique aspects of imaging from An important concept when thinking about an informatics perspective. Chapter 20 , on the imaging from an informatics perspective is that other hand, describes the use of images and image images are an unstructured data type ; as such, processing in various applications, particularly while machines can readily manage the raw those in radiology since that fi eld places the great- image data in terms of storage/retrieval, they can- est demands on imaging methods. not easily access image contents (recognize the The topics covered by this chapter and Chap. type of image, annotations made on the image, or 20 comprise the growing discipline of biomedi- anatomy or abnormalities within the image). In cal imaging informatics (Kulikowski 1997 ), a this regard, biomedical imaging informatics subfi eld of biomedical informatics (see Chap. 1 ) shares much in common with natural language that has arisen in recognition of the common processing (NLP; Chap. 8 ). In fact, as the meth- issues that pertain to all image modalities and ods of computationally representing and process- applications once the images are converted to ing images is presented in this chapter, parallels digital form. Biomedical imaging informatics is a to NLP should be considered, since there is syn- dynamic fi eld, recently evolving from focusing ergy from an informatics perspective. purely on image processing to broader informat- As in NLP, a major purpose of the methods of ics topics such as representing and processing the imaging informatics is to extract particular infor- semantic contents (Rubin and Napel 2010 ). At mation; in biomedical informatics the goal is the same time, imaging informatics shares com- often to extract information about the structure of mon methodologies and challenges with other the body and to collect features that will be useful domains in biomedical informatics. By trying to for characterizing abnormalities based on mor- understand these common issues, we can develop phological alterations. In fact, imaging provides general solutions that can be applied to all detailed and diverse information very useful for images, regardless of the source. characterizing disease, providing an “imaging 9 Biomedical Imaging Informatics 287 Image Image Acquisition Management/ Storage Image Content Representation Image Interpretation and Image Computer Reasoning Processing Fig. 9.1 The major topics in biomedical imaging infor- image content representation, management/storage of matics follow a workfl ow of activities and tasks com- images, image processing, and image interpretation/com- mencing with include image acquisition, followed by puter reasoning phenotype” useful for characterizing disease, While we seek generality in discussing bio- since “a picture is worth a thousand words.1 ” medical imaging informatics, many examples in However, to overcome the challenges posed by this chapter are taken from a few selected the unstructured image data type, recent work is domains such as brain imaging, which is part of applying semantic methods from biomedical the growing fi eld of neuroinformatics (Koslow informatics to images to make their content and Huerta 1997 ). Though our examples are spe- explicit for machine processing (Rubin and Napel cifi c, we attempt to describe the topics in generic 2010 ). Many of the topics in this chapter there- terms so that the reader can recognize parallels to fore involve how to represent, extract and charac- other imaging domains and applications. terize the information that is present in images, such as anatomy and abnormalities. Once that task is completed, useful applications that pro- 9.2 Image Acquisition cess the image contents can be developed, such as image search and decision support to assist In general, there are two different strategies in with image interpretation. imaging the body: (1) delineate anatomic struc- ture (anatomic/structural imaging), and (2) deter- 1 Frederick Barnard, “One look is worth a thousand mine tissue composition or function (functional words,” Printers’ Ink, December, 1921. imaging) (Fig. 9.2 ). In reality, one does not 288 D.L. Rubin et al. Radiography PET-CT CT MRI US PET Spatial resolution (anatomic detail) VLA HLA SA Planar NM ED ES Functional information (tissue composition) Fig. 9.2 The various radiology imaging methods differ mation depicted (which represents the tissue composi- according to two major axes of information of images, tion—e.g., normal or abnormal). A sample of the more spatial resolution (anatomic detail) and functional infor- common imaging modalities is shown choose between anatomic and