Elastic Registration of Multimodal Prostate MRI and Histology Via

Home , Image registration

Elastic registration of multimodal prostate MRI and histology via multiattribute combined mutual information Jonathan Chappelow Department of Biomedical Engineering, Rutgers University, Piscataway, New Jersey 08854 B. Nicolas Bloch Department of Radiation Oncology, Boston Medical Center, Boston, Massachusetts 02118 Neil Rofsky, Elizabeth Genega, Robert Lenkinski, and William DeWolf Beth Israel Deaconess Medical Center, Boston, Massachusetts 02115 ͒ Anant Madabhushia Department of Biomedical Engineering, Rutgers University, Piscataway, New Jersey 08854 ͑Received 17 December 2010; revised 5 February 2011; accepted for publication 10 February 2011; published 21 March 2011͒ Purpose: By performing registration of preoperative multiprotocol in vivo magnetic resonance ͑MR͒ images of the prostate with corresponding whole-mount histology ͑WMH͒ sections from postoperative radical prostatectomy specimens, an accurate estimate of the spatial extent of prostate cancer ͑CaP͒ on in vivo MR imaging ͑MRI͒ can be retrospectively established. This could allow for definition of quantitative image-based disease signatures and lead to development of classifiers for disease detection on multiprotocol in vivo MRI. Automated registration of MR and WMH images of the prostate is complicated by dissimilar image intensities, acquisition artifacts, and nonlinear shape differences. Methods: The authors present a method for automated elastic registration of multiprotocol in vivo MRI and WMH sections of the prostate. The method, multiattribute combined mutual information ͑MACMI͒, leverages all available multiprotocol image data to drive image registration using a multivariate formulation of mutual information. Results: Elastic registration using the multivariate MI formulation is demonstrated for 150 corresponding sets of prostate images from 25 patient studies with T2-weighted and dynamic-contrast enhanced MRI and 85 image sets from 15 studies with an additional functional apparent diffusion coefficient MRI series. Qualitative results of MACMI evaluation via visual inspection suggest that an accurate delineation of CaP extent on MRI is obtained. Results of quantitative evaluation on 150 clinical and 20 synthetic image sets indicate improved registration accuracy using MACMI compared to conventional pairwise mutual information-based approaches. Conclusions: The authors’ approach to the registration of in vivo multiprotocol MRI and ex vivo WMH of the prostate using MACMI is unique, in that ͑1͒ information from all available image protocols is utilized to drive the registration with histology, ͑2͒ no additional, intermediate ex vivo radiology or gross histology images need be obtained in addition to the routinely acquired in vivo MRI series, and ͑3͒ no corresponding anatomical landmarks are required to be identified manually or automatically on the images. © 2011 American Association of Physicists in Medicine. ͓DOI: 10.1118/1.3560879͔

Key words: mutual information, elastic registration, prostate cancer, in vivo, MACMI, magnetic resonance, histology

I. BACKGROUND AND MOTIVATION the ability to use different acquisition protocols to capture orthogonal sources of information, including functional Recently, magnetic resonance ͑MR͒ imaging ͑MRI͒ has ͓ ͑ ͔͒ ͓ emerged as a promising modality for detection of prostate dynamic-contrast enhanced DCE , metabolic magnetic ͑ ͔͒ ͓ cancer ͑CaP͒, with several studies showing that 3 T endorec- resonance spectroscopy MRS , vascular diffusion ͑ ͔͒ ͑ ͒ tal in vivo T2-weighted ͑T2-w͒ imaging yields significantly weighted imaging DWI , and structural T2-w attributes. higher contrast and resolution compared to ultrasound Since multiple protocols can be acquired in the same scan- ͑U.S.͒.1 For example, Fig. 1͑a͒ shows a typical in vivo U.S. ning session, little additional setup time is required. image of a prostate, in which internal anatomical details, The use of multiprotocol MRI for CaP diagnosis has been such as the urethra, ducts, and hyperplasia, are barely dis- shown to improve detection sensitivity and specificity com- cernible, while in the segmented T2-w MR image shown in pared to the use of a single MR imaging protocol.2–4 Previ- Fig. 1͑b͒, internal anatomical details within the prostate are ous studies have demonstrated improved CaP detection sen- clearly visible. An additional advantage offered by MRI is sitivity and specificity by simultaneous use of multiple MRI

FIG.1.͑a͒ Ultrasound imagery of the prostate provides poor soft tissue resolution, while ͑b͒ high resolution MRI ͑ex vivo image shown͒ shows internal anatomical details of the prostate with greater clarity. Ground truth for CaP extent is obtained only through histopathologic analysis of ͑c͒ the corresponding hematoxylin and eosin stained tissue section. The histopathologic CaP extent on ͑c͒ can be mapped onto the MRI in ͑b͒ by either ͑d͒ manually labeling the MRI using histology as a visual reference or ͑e͒ automatically mapping CaP extent from ͑c͒ via image registration. Note that the morphology of the CaP extent is better preserved in the mapping from ͑c͒ onto ͑e͒ as compared to ͑d͒. protocols, including DCE and T2-w MRI,5 MRS and T2-w sembles the histopathological ground truth for CaP extent in MRI,6 and DWI with both T2-w ͑Ref. 7͒ and DCE MRI.8 Fig. 1͑b͒ compared to the manually annotated region shown Since the current clinical diagnostic protocol involves no in Fig. 1͑d͒. Thus, with an accurate registration technique, image-based detection of CaP, the ability to utilize in vivo CaP extent on MRI can be established with greater accuracy, multiprotocol diagnostic images for detection and localiza- efficiency, and consistency compared to manual labeling. tion of CaP in vivo would have clear implications for ͑1͒ The registration of images from different modalities such noninvasive image-based screening, ͑2͒ targeted biopsies, as histology and radiology is complicated on account of the and ͑3͒ conformal radiation therapy. vastly different image characteristics of the individual mo- If the spatial extent for CaP on multiprotocol in vivo ra- dalities. For example, the appearance of tissue and anatomi- diological imaging can be accurately delineated, it may then cal structures ͑e.g., hyperplasia, urethra, and ducts͒ on MRI be possible to define specific imaging parameters with the and histology are considerably different, as may be appreci- greatest diagnostic accuracy in reliably characterizing CaP ated from Figs. 1͑b͒ and 1͑c͒. The shape of the WMH is also on in vivo clinical, radiologic images. The definition of such significantly altered due to uneven tissue fixation, gland slic- image signatures would be invaluable in building ͑a͒ a ing, and sectioning, resulting in duct dilation, gland deforma- computer-assisted disease detection system6,9–11 or ͑b͒ spatial tion, and tissue loss. Traditional intensity-based similarity disease atlases which could serve as training and educational measures, such as mutual information ͑MI͒, are typically in- tools for medical students, radiology residents, and fellows. adequate to robustly and automatically register images from However, direct annotation of disease extent on MRI is often two such significantly dissimilar modalities. There have been challenging even for experienced radiologists. Thus, to reli- several efforts to complement intensity information with al- ably ascertain the extent of CaP on in vivo radiological im- ternative image information, including image gradients,20 co- ages, it is necessary to utilize ex vivo tissue specimens, upon occurrence information,21 and color22 and image which “ground truth” estimates of CaP extent may be estab- segmentations23 in conjunction with MI variants, specifically lished by histopathologic inspection. ͓In the context of pa- adapted to incorporate these additional channels of informa- tients diagnosed with CaP and scheduled for radical prostate- tion. Similar to the use of calculated features to complement ctomy ͑RP͒, in several centers in the United States, image intensity, it may also be advantageous to leverage ad- preoperative imaging is performed to identify presence of ditional imaging protocols that may be available. For in- extracapsular spread.12͔ Figure 1͑c͒ shows a whole-mount stance, multiprotocol MR imaging is part of standard clinical histology ͑WMH͒ section of a RP specimen on which can- practice at a number of medical centers for disease diagnosis cerous tissue has been manually annotated, following micro- and treatment.24,25 These additional channels may provide scopic examination of the excised gland. complementary structural, metabolic, and functional data to Spatial correlation of diseased regions on histology and complement image intensity for the registration process. MRI may be performed by ͑a͒ visually identifying and label- In this paper, we present an information theoretic ap- ing corresponding structures on each modality9,13–15 or ͑b͒ proach, multiattribute combined mutual information using a semiautomated or fully automated image registration ͑MACMI͒, to simultaneously utilize all available imaging procedure.16–19 For example, the spatial extent of CaP on channels, such as registered multiprotocol imagery ͑or image MRI, obtained by manually labeling the in vivo image while features calculated from the original images͒, in the registra- visually referencing the histology, is shown in Fig. 1͑d͒.On tion of several images. In this work, we demonstrate the the other hand, the disease extent established by the elastic application of MACMI for establishing spatial extent of CaP registration of the MR and histology images in Figs. 1͑b͒ and on radiological imaging via registration of annotated ex vivo 1͑c͒ is shown in Fig. 1͑e͒. Note that the shape of the disease histology sections with corresponding multiprotocol in vivo mask mapped onto the MRI in Fig. 1͑e͒ more closely re- MRI.

Medical Physics, Vol. 38, No. 4, April 2011 2007 Chappelow et al.: Multiattribute combined mutual information 2007

II. PREVIOUS WORK WMH of the prostate has also been recently addressed.18,30,35,38 Lee et al.30 performed histopathologic The registration of prostate images is important for ͑a͒ the validation of CaP estimates on CT and MRI by 2D elastic planning, guidance, and retrospective evaluation of radio- registration of histology sections with CT and MRI. Manu- 26–29 therapy procedures; ͑b͒ the fusion of diagnostic images ally identified control points ͑anatomical landmarks͒, placed 30–33 for improved CaP detection accuracy; and ͑c͒ the auto- primarily along the gland boundary, were used to define a mated annotation of radiological images via histopathologic TPS interpolant. In a rat brain study, Meyer et al.35 also 18,34,35 correlation. Intramodality prostate image registration leveraged available block face photographs of the gland prior 26 has been utilized by Bharatha et al. for image-guided pros- to sectioning ͑similar to Ref. 34͒ to generate a histology tate surgery via elastic registration of preoperative and intra- volume for 3D registration. Their approach also utilized an 27 operative prostate MRI and by Foskey et al. for monitoring intermediate ex vivo MRI series, to which WMH was aligned therapy related changes over time by registration of serial CT via a TPS-based approach using manually selected initial images. Lee et al.30 investigated the use of multimodal pros- control points, followed by MI-driven refinement of the co- tate images ͑MRI, CT, and SPECT͒ to characterize CaP, per- ordinates of the control points. Subsequently, ex vivo MRI forming automated multimodal registration of MRI to was registered to in vivo MRI, thus indirectly aligning the in SPECT via MI and manual registration of MRI to CT using vivo MRI and histology slices of the rat brains. Park et al.18 a graphical user interface. Automated rigid registration of extended this approach to the human prostate and to include MRI with CT has been addressed by several groups,31–33 multiprotocol ͑T2-w and DWI͒ in vivo MRI and PET, again while automated elastic registration has only recently been using block face photographs and ex vivo MRI as an inter- addressed by Park et al.36 using a spatially constrained mediate. While the works of Park18 and Meyer35 successfully B-spline. Methods for the registration of MRI and 3D tran- address the need to rely on approximate slice correspon- srectal ultrasound for real-time MRI-guided prostate biopsy dences, neither block face photographs nor ex vivo MRI of have been presented by both Xu et al.28 and Singh et al.29 prostate specimens are usually available in the course of the The unique set of challenges associated with the registra- clinical workflow. This might also explain why only two tion of ex vivo histology and multiprotocol MRI of the pros- patient studies were employed in Ref. 18 and one rat brain tate has begun to be addressed by several recent studies; slide in Ref. 35. these studies have, however, primarily been in the context of Alignment of more than two images or volumes repre- high resolution ex vivo MRI11,19,34,37 utilized a thin plate senting very different structural or functional attributes of the spline ͑TPS͒ to model the elastic 2D deformations of histol- same object is not well studied. One approach to the regis- ogy to ex vivo MRI of prostate specimens using control tration of multiple images is to take a groupwise ͑GW͒ ap- points. However, “block face” photographs of thick tissue proach, whereby all images are simultaneously aligned, usu- sections of the prostate, taken prior to microtome slicing and ally to some reference image or coordinate frame. The slide preparation, were used to facilitate correction of the limitations of fully GW approaches are that they either ͑1͒ nonlinear tissue deformations and in the creation of a histol- involve optimization problems with many degrees of free- ogy volume. While these photographs allowed Park et al.34 dom arising from multiple simultaneous transformations or to overcome the nonlinear deformations to histology and ad- ͑2͒ are limited to images with similar intensity and/or defor- dress the issue of slice correspondences ͑identifying a one- mation characteristics. In the GW registration method pre- to-one relationship between histology sections and slices in sented by Bhatia,39 all images contribute to the same histo- the MRI volume͒, such photographs are not generally ac- gram used for entropy calculation. This limits the technique quired as part of routine clinical practice. Recently, Ou et to images of the same modality. On the other hand, the GW al.19 aligned ex vivo MRI and histology sections of a prostate method of Studholme40 utilizes a high dimensional distribu- specimen with the aid of precise cancer labels on both mo- tion suitable for multimodal data, but the use of a dense dalities to improve their objective function for registration. deformation field requires a constraint that penalizes defor- As part of an integrated registration and segmentation strat- mations that deviate from an average deformation. However, egy, the required cancer extent on MRI was established via a in the context of large deformation fields, such as might be pixelwise supervised classifier. The authors, however, did not present between ex vivo and in vivo images, this technique is address how classifier errors would affect registration accu- restrictive. Other methods require repeated refinement of in- racy. While high resolution ͑4T͒ ex vivo MRI provided suf- dividual transformations prior to convergence.41 More re- ficient segmentation accuracy in Ref. 19, it is not clear that cently, Balci42 performed simultaneous interpatient registra- disease extent can reliably be established on in vivo clinical tion of a large number ͑50͒ of brain MRI scans using a sum images. Zhan et al.37 also performed registration of ex vivo of univariate ͑1D͒ entropy values ͑“stack entropy”͒ calcu- MRI and histology sections using pairs of automatically de- lated at every pixel location. However, since this cost func- tected control points on each modality. However, automated tion requires a large number of images to calculate entropy at identification of a large number of landmark pairs across ex each pixel location, it is suited only for the registration of vivo WMH and in vivo MRI ͑of lower image resolution and very large populations of images from the same modality, as quality compared to the ex vivo MRI used in Ref. 37͒ may opposed to a smaller number of multimodal images from a not be feasible. single patient. Thus, while a GW approach is generally pref- The registration of clinical in vivo radiologic images with erable to several pairwise ͑PW͒ registration steps, GW meth-

Medical Physics, Vol. 38, No. 4, April 2011 2008 Chappelow et al.: Multiattribute combined mutual information 2008

A A A 2

C B C B C B

(a)(b) (c)

FIG. 2. Registration of an ex vivo prostate histology ͑A͒ image to corresponding in vivo T2-w ͑B͒ and T1-w ͑C͒ MR images can be achieved in different ways. Two possible approaches using PW image registration involve ͑a͒ PW alignment of histology to each individual MRI protocol ͑A→B and A→C͒ or ͑b͒ alignment of multiprotocol MRI ͑C→B͒ and alignment of histology to just T2-w MRI ͑A→B͒. In the latter case, T1-w MRI would be in implicit alignment with histology at the end of the two registration steps. Alternatively, ͑c͒ a multiattribute image registration scheme involves initial PW alignment of images from the same modality ͑T1-w and T2-w MRI͒ as in ͑b͒, followed by alignment of histology to a multiattribute image comprising the registered multiprotocol MRI via a similarity measure specifically defined for high dimensional data. ods can be restrictive or computationally prohibitive. the newly aligned images could be considered in unison ͑as a Given the challenges and constraints of GW registration “multiattribute” image͒ to drive the registration with A. Such approaches, simple sequential PW registration steps are most an approach would exploit the fact that C and B are ͑a͒ in commonly performed to bring multiple images from differ- implicit alignment and ͑b͒ represent different and informa- ent modalities into alignment. This was the method of choice tive image attributes ͑in this case, structural and functional͒. for the prostate work in Refs. 30 and 38, where several mo- This approach is akin to previous studies that have used ad- dalities are registered in steps using two images at a time. ditional textural and gradient feature images20 to comple- Figures 2͑a͒ and 2͑b͒ illustrate two possible approaches to ment image intensity in order to improve image registration. PW registration of an ex vivo prostate histology ͑A͒ image to Multivariate formulations of MI have been shown to be corresponding in vivo T2-w MR ͑B͒ and in vivo T1-w ͑single useful in incorporating multiple image attributes ͑e.g., ͒͑C͒ ͑ ͒ 21,38 time point of a DCE series MR images. Figure 2 a texture͒. Thus, multivariate MI may also be applied in A illustrates the case where image is independently regis- the context of applications where multiprotocol imaging B C ͑ ͒ tered to both and . Figure 2 b illustrates a scenario ͑e.g., T2-w and T1-w MRI͒ needs to be registered to another where the multiprotocol MR images are coregistered by the modality ͑e.g., histology͒. C B A B alignment of to and is registered to just , thus bring- The novel contribution of this work is a formal quantita- A B C ing into alignment with both and . For this particular tive image registration framework, which we refer to as set of multimodal images, the approach illustrated in Fig. MACMI. MACMI allows for incorporation of multiple mo- ͑ ͒ ͑ ͒ 2 b is preferable to that shown in Fig. 2 a since the align- dalities, protocols, or even feature images in an automated ment between multiprotocol MRI is less complicated com- registration scheme, facilitated by the use of multivariate MI. pared to the multimodal alignment of ex vivo histology and MACMI is distinct from previous GW approaches in that it in vivo MRI. The latter approach involves dealing with handles images that can significantly vary in terms of image highly elastic deformations and dissimilar intensities. In both intensities ͑e.g., multimodal data͒ and deformation character- instances ͓illustrated in Figs. 2͑a͒ and 2͑b͔͒, the two registra- istics ͑e.g., in vivo to ex vivo͒. Additionally, it involves a tion steps are independent and utilize only two images at a simple ͑low degree of freedom͒ optimization procedure time. whereby individual image transformations are determined in sequence. The use of an information theoretic similarity III. NOVEL CONTRIBUTIONS AND SIGNIFICANCE measure is central to the ability of MACMI to handle ͑1͒ Both PW approaches illustrated in Figs. 2͑a͒ and 2͑b͒ multimodal data, which may contain nonlinear relationships consider only two images at a time. Hence, they exploit only between the image intensities of different modalities, and ͑2͒ a fraction of the available data in driving each registration high dimensional ͑multiattribute͒ observations, which may step. Further, in subsequent alignment steps, it is necessary contain redundancies between the attributes that can be dis- to select only a single image from the set of coregistered counted via joint entropy. Finally, by employing a sequential images for use as a reference. A more effective approach is to approach to the alignment of multiple images within the mul- exploit all the information acquired from prior alignment tiattribute representation, each successive optimization pro- steps to drive the subsequent registration operations. As il- cedure remains as simple ͑in terms of degrees of freedom͒ as lustrated in Fig. 2͑c͒, following registration of C to B, both with the conventional PW registration.

Medical Physics, Vol. 38, No. 4, April 2011 2009 Chappelow et al.: Multiattribute combined mutual information 2009

Cancer Gold Standard

(d)

(a) Multi-attribute T2-w MRI Registration (MACMI) Multiprotocol Registration (c) Multi-attribute MRI (e) (Co-registered Multiprotocol MRI)

(b) T1-w MRI

FIG. 3. Establishing disease signatures on in vivo multiprotocol MRI using MACMI for registration of ͑a͒ WMH sections to corresponding ͑d͒ T2-w and ͑e͒ T1-w MRI. Alignment of the T2-w and T1-w MRI allows generation of ͑c͒ the multiattribute MRI comprised of coregistered multiprotocol MRI. MACMI is used to align ͑a͒ WMH to ͑c͒ the multiattribute MRI. CaP extent on the ͑b͒ elastically registered WMH is mapped directly onto both MR images in ͑c͒.

In this work, we evaluate MACMI in the context of a T2-w protocols are simultaneously considered. MACMI is clinical problem involving multiprotocol ͑T2-w and DCE͒ compared to PW MI-based approaches where T1-w and MRI of the prostate and WMH sections, the sections being T2-w MR images are individually registered with PD MRI. digitized following gland resection. Registration of WMH to corresponding multiprotocol MRI is then performed to map IV. METHODS CaP extent from ex vivo WMH ͑previously delineated by pathologists͒ onto in vivo MRI. This procedure involves ͑1͒ IV.A. Theory on mutual information and MACMI initial affine alignment of the T2-w and DCE ͑T1-w͒ images IV.A.1. Mutual information between scalar-valued ͓Figs. 3͑d͒ and 3͑e͔͒ using MI to generate a multiattribute images ͓ ͑ ͔͒ ͑ ͒ MR image Fig. 3 c , followed by 2 multimodal elastic Equation ͑1͒ below is a common formulation of MI for a registration of WMH with the multiattribute MRI. ͑ ͒ A A pair of images or random variables 1 , 2 in terms of Sh- Our scheme for registration of in vivo MRI and ex vivo annon entropy. WMH of the prostate is distinct from previous related 18,30,35 ͑A A ͒ ͑A ͒ ͑A ͒ ͑A A ͒ ͑ ͒ efforts in that ͑1͒ information from all in vivo imaging I2 1, 2 = S 1 + S 2 − S 1, 2 , 1 protocols is being utilized simultaneously to drive the pro- ͑A A ͒ where I2 1 , 2 describes the interdependence of two vari- cess of automated elastic registration with histology; ͑2͒ no 20 ͑A A ͒ ables or intensity values of a pair of images. As I2 1 , 2 additional, intermediate ex vivo radiology or gross histology A A increases, the uncertainty about 1 given 2 decreases. images need be obtained in addition to the clinically acquired Thus, it is assumed that the global MI maximum will occur ͑ ͒ in vivo MRI series; and 3 no point correspondences are at the point of precise alignment, when maximal uncertainty required to be identified manually or automatically. about intensities of A can be explained by A . For the registration of 150 corresponding sets of prostate 1 2 images from 25 patient studies with T2-w and DCE MRI, we quantitatively compare MACMI to a PW registration ap- IV.A.2. Mutual information between high proach using conventional MI ͓Fig. 2͑b͔͒. For 15 patients for dimensional „multiattribute… images which apparent diffusion coefficient ͑ADC͒ MRI was also The conventional MI formulation can be extended to high acquired, we further demonstrate MACMI for including the dimensional observations by combining the multiple dimen- third MR protocol in the elastic registration of histology with sions or attributes via high order joint entropy calculations. all three MRI series for 85 sets of images ͑in vivo T2-w, DCE We refer to this application of MI as MACMI to distinguish and ADC MRI slices, and ex vivo WMH sections͒. We also it from conventional applications of MI and higher order MI ء quantitatively evaluate MACMI on a synthetic brain MRI and denote it as I2. Unlike the more familiar higher order MI 43 ͑ Ն ͒ study from BrainWeb, whereby T1-w and T2-w MR im- In ,n 2 , the goal of MACMI is not to measure only the ages are registered to PD MRI, where both the T1-w and intersecting information between multiple sources

Medical Physics, Vol. 38, No. 4, April 2011 2010 Chappelow et al.: Multiattribute combined mutual information 2010

͑A A ͒ ͒ ␧ ͑ ͒ 1 ,..., n , but to quantify the combined predictive value semble into an d+n -dimensional ensemble by concat- ͑ ͓A A ͔͒ ␧ of one multivariate source e.g., 1 ,..., n with respect enation with a d-dimensional ensemble d. We also denote ͑ ͓B B ͔͒ Z ⇐␧ ͑ to another e.g., 1 ,..., n . Here we introduce the notion j as the assignment of each of the m dimensions in- ͒ ␧ Z of an image ensemble as the concatenation of n intensity- tensity images in to the existing m total members of j ͑I I ͒ ͑ ͑ valued images 1 ,..., n into an n-dimensional multiat- independent of the organization of images within the family ͒ ͓I I ͔ ͒ Z tribute image, denoted as 1 ,..., n . In the simplest case, structure , thus replacing or updating j with the contents of A ␧ ͒ء ͑ the MI I2 that a single image 1 shares with an ensemble . B B of two other images, 1 and 2,is Algorithm MACMIreg ء I ͑A ,͓B ,B ͔͒ = S͑A ͒ + S͑B ,B ͒ − S͑A ,B ,B ͒. ͑2͒ ͕ ͖ Ն 2 1 1 2 1 1 2 1 1 2 Input: Z= Z1 ,...,Zn , n 1. B B Output: ␧. By considering 1 and 2 as simultaneously measured semi- independent variables in the multidimensional ensemble Auxiliary Data Structures: Index k, j,␣; Image ␧ ͓B B ͔ B B ensemble 0. 1 , 2 , any dependence that exists between 1 and 2 may begin ء be discounted and I2 remains bounded by the smaller of ͑A ͒ ͑B B ͒ 0. for j=1 to ndo S 1 and S 1 , 2 . The generalized form of MI between 1. k=͉Z ͉; ␧A ͓A A ͔ j the n-dimensional ensemble = 1 ,..., n with the Ͼ B n 2. if k 1 then ␧ ͓B B ͔ ␧ ͑ ͒ m-dimensional ensemble m = 1 ,..., m is 3. Obtain ensemble 0 =MACMIreg Zj ; ; A B A B A B 4. Update Z ⇐␧ ء I ͑␧ ,␧ ͒ = S͑␧ ͒ + S͑␧ ͒ − S͑␧ ,␧ ͒. ͑3͒ j 0 2 n m n m n m 5. endif; Thus, MACMI accomplishes fusion of the multiple dimen- 6. endfor; sions of a multiattribute image, allowing only intersecting 7. Initialize ␧ as an empty ensemble; A B ␧←͓Z Z ͔ ʈ ʈ information between two such images ͑e.g., ␧ and ␧ ͒ to be 8. 1 ,..., k , k= Z1 ; A B n m ␣ ;␧ ,␧ ͒ is discussed in Sec. 9. =k+1͑ءquantified. Calculation of I 2 n m 10. for j=2 to ndo IV C. ʈ ʈ 11. k= Zj ; ␧ ͓Z Z ͔ 12. 0 = ␣ ,..., ␣+k ; ͔͒͒ ␧ ͑␧͑ء ͓ IV.B. Framework for registration of multiple images 13. Obtain T=argmaxT I2 ,T 0 ; ␧ ͑␧ ͒ ͓Z˜ Z˜ ͔ using MACMI 14. Obtain ˜ 0 =T 0 = ␣ ,..., ␣+k ; ␧←␧ 15. ˜ 0; In Sec. IV B 1, we present a generalized algorithm 16. ␣=␣+k+1; ͑ ͒ MACMIreg for performing registration of m images 17. endfor; Z Z 1 ,..., m in a specific order. The order is specified using a end hierarchical organization of the images within a family of Lines 1–6 of MACMIreg use recursive calls to MAC- sets Z and by progressively aligning and accumulating the MIreg to register the images within each Zj containing more registered images into an single ensemble ␧. In Sec. IV B 2 ͑ ͒ than one image. When MACMIreg Zj is executed on line 3, we illustrate the operation of the algorithm for four images the algorithm is recursively instantiated in order to coregister ͑Z Z Z Z ͒ Z Z 1 , 2 , 3 , 4 , where Z is structured to register 1 with 2 the images within the subset Z and any of its subsets, re- Z Z j and 3 with 4, prior to the alignment of the two resulting turning the registered images within ensemble ␧. Line 4 then ensembles. updates each Zj by replacing its constituent elements with the coregistered member images contained within ␧. Lines IV.B.1. Algorithm 7–17 of MACMIreg perform the registration between the Ն multiattribute images generated from each Zj, each of which Consider a family of sets Z that contains m 2 images ͑ ͒ Z Z Յ now comprise only coregistered images or a single image 1 ,..., m distributed throughout n m ordered subsets Zj ͑ ෈ ͒ ͕ ͖ ͑ ഫ ͕Z Z ͖ following lines 1–6 of the algorithm. A spatial transforma- j I , where I= 1,...,n , i.e., j෈IZj = 1 ,..., m and ␧ പ ͒ ͑ ෈͕ ͖͒ tion T of the current moving image ensemble 0 into align- j෈IZj = . Each subset Zj j 1,...,n may also be a ␧ ͑ ͒ ment with the stationary growing ensemble is determined family i.e., have subsets of its own or simply an ordered set ␧ on line 13. The registered ensemble ˜ 0, obtained via T on of registered images. For example, if Zj ͑ ͒ ͑ ͒ ͑ ͒ ͑ ͒ line 14, is then combined with ␧ on line 15. The algorithm =͕͕Z j ,Z j ͖,͕Z j ͖,͕Z j ͖͖, we define Z as a family of ͉Z ͉ 1 2 3 4 j j Z ʈ ʈ continues to align each subsequent j with the expanding =3 subsets, containing a total of k= Zj =4 images. We fur- ␧ ␧ ͗ ͘ reference ensemble . ther denote the ensemble of all k images in Zj as = Zj ͓Z͑j͒ Z͑j͔͒ = 1 ,..., k . By organizing the m images into a hierar- chy of subsets within the family Z, the order in which the IV.B.2. Instance of MACMI for registration of four images are registered and combined into multiattribute im- images ages is determined. The procedure for alignment of all im- The operation of the MACMI algorithm is illustrated in ͑ ͒ ␧ ages within and between each Zj into a single ensemble Fig. 4 for a scenario involving the registration of four images ͑Z Z Z Z ͒ Z Z of registered images is described in the following recursive 1 , 2 , 3 , 4 , of which 3 and 4 are designated to be ␧←␧ Z Z algorithm MACMIreg. Here we define the notation d as coregistered prior to alignment with 1 and 2. In this ge- ͑ Z Z the expansion of an n-dimensional multiattribute image en- neric example, 1 and 2 could represent images from two

Medical Physics, Vol. 38, No. 4, April 2011 2011 Chappelow et al.: Multiattribute combined mutual information 2011

MACMIreg(Z = ff Z 1g; fZ2g; ff Z 3g; fZ4ggg ) MACMIreg(Z = ff Z 3g; fZ4gg )

Z Z

jZ3j > 1(lines1-2)

Z Z Z Z Z 1 fZ1gfZ2 2gfZ3 ff Z 3g; fZ4gg 1 3gfZ2 4g

Line 13 inputs: Line 13 inputs: line 3 1: " =[Z1]; "0 =[Z2]; " =[Z3]; "0 =[Z4]; ~ ^ 2: " =[Z1; Z2]; "0 = MACMIreg(Z3); Output: " =[Z3; Z4] ~ ~ ~ Output: " =[Z1; Z2; Z3; Z4] (a) (b)

͑ ͒ ͑Z Z ͒ ͑ ͕͕Z ͖ ͕Z ͖ ͕͕Z ͖ ͕Z ͖͖͖͒ FIG.4. a Graphical representation of the organization of four images 1 ,..., 4 within a family of image sets Z= 1 , 2 , 3 , 4 and the ͑ ͉ ͉ ͒ ͑ ͒ application of the MACMIreg algorithm for alignment of all four images. Since only Z3 contains subsets i.e., Z3 =2 , line 3 of MACMIreg in a begins a ͑ ͒ ͕͕Z ͖ ͕Z ͖͖ ͑ ͒ Z Z ͓Z Zˆ ͔ new instance of the algorithm in b with Z= 3 , 4 as the input. The instance in b brings 3 and 4 into alignment and returns the ensemble 3 , 4 ͑ ͒ ͑ ͒ Z Z ͑␧ ͓Z Zˆ ͔͒ to the instance in a . The instance in a ﬁrst brings 1 and 2 into alignment and then align the ensemble of registered images from Z3 0 = 3 , 4 with ␧ ͑␧ ͔͒͒ ␧͑ء ͓ ͒ ͑ ͔ ˜␧ ͓Z Z the registered images of Z1 and Z2 = 1 , 2 . At each registration step line 13 , a transformation T is determined by argmaxT I2 ,T 0 and is then ␧←␧ ͑␧ ͒͑ ͒ ͑ ͒ ␧ ͓Z Z˜ Z˜ Z˜ ͔ expanded by ˜ 0 =T 0 lines 14 and 15 of MACMIreg . The output of a , containing all of the coregistered images in Z,is = 1 , 2 , 3 , 4 .

Z Z different modalities, such as CT and PET, and 3 and 4 order of multiattribute image construction. Implementation could represent multiprotocol images from the same modal- of MACMI within a complete registration framework is de- ity such as T1-w and proton density ͑PD͒ MRI that may be in scribed in Sec. IV C below. proximal alignment through hardware configuration or through prior application of a registration routine. IV.C. Requirements for implementation of MACMI The operation of MACMIreg͑Z͒ for Z MACMI can be utilized to leverage multiple image =͕͕Z ͖,͕Z ͖,͕͕Z ͖,͕Z ͖͖͖ begins by registration of images 1 2 3 4 sources in nearly any registration application by selecting the within each Z ͑j෈͕1,2,3͖͒, where only Z =͕͕Z ͖,͕Z ͖͖ j 3 3 4 following components based on domain requirements: contains more than one image. Thus, as in Fig. 4͑b͒, MACMIreg͑Z ͒ is called to register Z to Z and update Z ͑1͒ MI estimation for high dimensional data: The most A B ء 3 4 3 3 ␧ ͓Z Zˆ ͔͑ ͒ straightforward approach to estimating I ͑␧ ,␧ ͒ is to with ensemble = 1 , 2 lines 3–4 of MACMIreg . Hav- 2 n m ͑ formulate the density estimates from high dimensional ing registered the images within each Zj lines 1–6 of MAC- MIreg͒, all images in Z are registered as in Fig. 4͑a͒ in two histograms. While histogram-based techniques are fea- steps ͑lines 7–17 of MACMIreg͒. At each registration step, sible and effective for up to four-dimensional observa- ␧ ␧ tions with appropriate bin size, as demonstrated in Refs. an optimal spatial transformation T of 0 to is determined ␧ ͑␧ ͔͒͒ ͑ ͒ ␧ 23 and 21 higher dimensionality necessitates an alternate͑ء ͓ by argmaxT I2 ,T 0 line 13 of MACMIreg and is ␧←␧ ͑␧ ͒͑ estimate of entropy or MI, such as those based on en- then expanded by ˜ 0 =T 0 lines 14–15 of MAC- ͒ ␧ ͓Z ͔ tropic spanning graphs or related quantities such as MIreg . Thus, Z2 is first registered to Z1, where = 1 and ␣-MI.44 ␧ =͓Z ͔, and ␧←˜␧ =Z˜ :=T͑Z ͒. Next, Z is registered to 0 2 0 2 2 3 ͑2͒ Image transformation model(s): Since MACMI only ͑ ͒ ␧ ͓Z Z˜ ͔ ␧ ͓Z Zˆ ͔ Z1 and implicitly Z2 , where = 1 , 2 and 0 = 3 , 4 dictates the construction of the objective function, ͓ ͑ ͔͒ ␧←␧ ͓Z˜ Z˜ ͔ the output of MACMIreg Z3 , and ˜ 0 = 3 , 4 : MACMI is agnostic to the deformation model. Further, ͓ ͑Z ͒ ͑Z ͔͒ ␧ ͓Z Z˜ Z˜ Z˜ ͔ = T 3 ,T 4 . The final output is = 1 , 2 , 3 , 4 , different deformation models may be used for each im- comprising all of the coregistered images in Z. age since the individual image transformations are per- Z Z ͑ Z Z ͒ The use of both 3 and 4 and both 1 and 2 in the formed in independent steps. ␧A ␧B͒͑ء ͒ ͑ ͒ ͑ final registration step has the following benefits: 1 Avoids 3 Optimization scheme to find a maximum of I2 n , m : Z Z ͑ potential ambiguity in choosing between 3 and 4 between If the analytical gradient can be derived, as demon- Z Z ͒ ͑ ͒ ␣ 1 and 2 and 2 potentially provides improved alignment strated for -MI in Ref. 45, an efficient stochastic gra- Z Z ͑Z Z ͒ versus use of just 3 or 4 1 or 2 individually. The dient descent method can be used. In the absence of ␧A ␧B͒͑ء advantage of MACMI is that it yields cumulative incorpora- analytical gradients of I2 n , m , methods including di- tion of all images, while allowing flexibility to choose the rect search ͑e.g., downhill simplex͒, quasi-Newton ͑e.g.,

Medical Physics, Vol. 38, No. 4, April 2011 2012 Chappelow et al.: Multiattribute combined mutual information 2012

TABLE I. Summary of the synthetic and clinical data sets registered by MACMI. Pixels are square.

Data set Description Modalities Dimensions Studies ͑images͒

Synthetic multiprotocol Ss brain MRI from BrainWeb T2-w, T1-w, PD MRI 181ϫ217 ͑1 mm pixel͒ 1 ͑20͒ Clinical multiprotocol c ϫ ͑ ͒ ͑ ͒ S1 MRI and histology of prostate In vivo T2-w, DCE MRI, ex vivo WMH T2-w MRI: 512 512 0.230–.280 mm pixel 25 150 Clinical multiprotocol c ϫ ͑ ͒ ͑ ͒ S2 MRI and histology of prostate ADC, T2-w, DCE MRI, ex vivo WMH T2-w MRI: 512 512 0.230–.280 mm pixel 15 85

ء T ͑Pd͔͒͒͑ ͓ ͒ Broyden–Fletcher–Goldfarb–Shanno , and other finite =argmaxT I2 2,T . Estimation of I2 and I2 was difference-based schemes can be employed. achieved using 2D and 3D probability density estimates obtained using histograms with 128 and 62 gray level bins, Specific implementation details employed in this work are respectively. The number of bins in each case were chosen described in the following section. empirically for reliable optimization of Eq. ͑4͒ using a Nelder–Mead simplex algorithm.47 V. EXPERIMENTAL DESIGN A summary of the clinical prostate and synthetic brain V.A.3. Registration evaluation image data sets investigated in this study is presented in For the synthetic data, quantitative evaluation of registra- Table I. tion accuracy can be performed easily since the correct coordinate transformation Tap is known. The magnitude of er- V.A. Synthetic multiprotocol brain MRI ror in the transformation Tco determined by registration can V.A.1. Data description be quantified in terms of mean absolute difference ͑MAD͒ ͓F ͑Tco͔͒ and root mean squared ͑RMS͒ error To quantitatively evaluate the performance of MACMI, MAD ͓F ͑Tco͔͒ from Tap. Both MAD and RMS error are com- we consider a synthetic registration task using a data set Ss RMS puted over the N total image pixels c in the common coor- comprising 20 2D multiprotocol ͑T1-w, T2-w, and PD͒ MRI dinate frame C of T1, T2, and P and can be expressed as slices from the BrainWeb simulated brain database.46 We denote the T1-w, T2-w, and PD MRI slices as T1, T2, and P, 1 ͑ co͒ ͚ ʈ co͑ ͒ ap͑ ͒ ʈ ͑ ͒ respectively. FMAD T = T c − T c L2 , 5 N c෈C

V.A.2. Registration Experiment 1 F ͑Tco͒ = ͱ ͚ ʈTco͑c͒ − Tap͑c͒ʈ2 , ͑6͒ Since synthetic brain MRI volumes T1, T2, and P are RMS L2 N c෈C initially in alignment, we apply a known nonlinear deforma- ap d ͉ ͉ ʈ ʈ 2 tion ͑T ͒ to P to generate an image P , for which misalign- where N= C and . L2 is the L -norm of a coordinate vector. ment from T1 and T2 is known. The objective of the regis- Further, the original P is compared directly to the resulting Pr 2 ͑ ͒ tration task is to recover the initial correct alignment via a using L distance DL2 as the similarity measure. corrective deformation ͑Tco͒. We denote the recovered P slice as Pr. Image transformation is implemented using an V.B. Clinical multimodal prostate MRI and histology ͑ ͒ elastic free form deformation FFD model with a hierarchi- V.B.1. Data description cal mesh grid spacing scheme, as described in Ref. 21. Three c mesh grid levels were defined for vertex spacings of approxi- We address the registration of two prostate data sets, S1 c ͑ mately ͕36, 27, 18͖ mm at each level ͑n and S2, comprising multimodal 3Tin vivo MRI and histol- x,y ͒ ͑ ͒ ෈͕͑6,5͒, ͑8,7͒, ͑12,10͖͒ moving control points on ogy and multiprotocol T2-w, DCE, and ADC MRI images ͑ ͒ c zero-padded images͒. MACMI is performed in a manner see Ref. 24 for details on acquisition . Set S1 comprises 150 similar to the scenario illustrated in Fig. 2͑c͒, whereby Pd corresponding ex vivo WMH sections with CaP delineated ͑ ͒ and their closest corresponding 3 T in vivo T2-w and DCE instead of WMH is registered to the multiattribute image c comprising the coregistered sections T1 and T2 via the re- MRI slices over all 25 patient studies. Set S2 is a subset of 15 ͑ ͒ c covered transformation patients 85 image sets in S1, for which DWI was also acquired and ADC maps calculated. We denote the T2-w MRI, ͒ ͑ ␧͑T T ͒ ͑Pd͔͒͒͑ء ͓ co TMACMI = argmax I2 1 2 ,T , 4 DCE MRI, ADC MRI, and WMH images as S, F, D, and H, T respectively. Each DCE MRI series comprises a sequence of where ␧͑T1T2͒ represents the ensemble ␧ of T1 and T2. In T1-w gradient echo MRI volumes, acquired following bolus order to compare to MACMI, registration is also performed injection of the gadopentetate dimeglumine contrast agent at using objective functions defined by the MI of ͑1͒ Pd with 0.1 mmol/kg of bodyweight. Two precontrast and five post- T1 and ͑2͒ Pd with T2. Thus, two additional Pr images are contrast T1-w MRI volumes were obtained at a temporal co ͓ ͑T ͑Pd͔͒͒ co obtained by TPW1 =argmaxT I2 1,T and by TPW2 resolution of 95 s. Maximal enhancement was generally ob-

Medical Physics, Vol. 38, No. 4, April 2011 2013 Chappelow et al.: Multiattribute combined mutual information 2013 tained at the third postcontrast time point ͑denoted as F3͒, H F˜ 3 ͓ ͑F˜ 3 e͑H͔͒͒ tional MI of with I2 ,T . We refer to these which was designated for use in the registration routines. No PW registration approaches as PW-T2 and PW-DCE, respec- ex vivo MRI or gross pathology photographs were acquired. e tively, and denote corresponding transformations as TMACMI, Following RP and prior to sectioning, the excised prostate e e TPW-T2, and TPW-DCE. was embedded in a paraffin block while maintaining the ori- c For data set S2, the 3D ADC volume for each patient is entation to keep the urethra perpendicular to the plane of also registered using a 3D affine transformation to the coreg- slicing. This procedure facilitates the identification of a cor- istered T2-w and DCE MRI volumes via a multiattribute responding in vivo 2D axial MRI slice for each 2D histology volume ͓i.e., the volume composed of axial images ␧͑SF˜ 3͔͒, slice. Preparation of the digitized WMH sections proceeds as hence generating a registered ADC volume comprising slices follows: ͑1͒ The excised prostate is cut into sections that are D˜ . Automated FFD registration of WMH to MRI is then 3–4 mm thick by slicing axial sections from the paraffin performed for each set of corresponding H and S slices by block using a circular blade, ͑2͒ a microtome is used to fur- F˜ 3 D˜ ther cut the sections into thin slices that are about 5 ␮m MACMI, which also considers and via ␧͑SF˜ 3D˜ ͒ e͑H͒͒͑ء thick, and ͑3͒ a single thin slice from each 3–4 mm thick I2 ,T . CaP extent is then mapped to each of section is chosen and digitally scanned. Each slide is then S, F˜ 3, and D˜ . PW registration using just H and D is not examined under a light microscope using up to 40ϫ apparent performed as the DWI protocol provides insufficient spatial magnification to identify and delineate the regions of CaP. As resolution and anatomical detail for multimodal correlation a result of this slide preparation process, spacing of the digi- based on image intensities alone. tal WMH slides is both coarse and irregular, ranging from 3–8 mm. Further, the nonlinear tissue deformations and artifacts introduced during microtome slicing are independent V.B.3. Registration evaluation between slices and no block face photographs or fiducial markers are available to correct the distortions. Therefore, it Since ground truth for alignment of the clinical prostate is not feasible to accurately construct a 3D histology volume data is not known or easily determinable, evaluation is per- ˜ 3 suitable for a 3D registration procedure without significant formed by calculating similarities of both S and F with 48,49 e ͑H͒ e ͑H͒ e ͑H͒ modifications to the established clinical routine. Thus, TMACMI , TPW-T2 , and TPW-DCE . To exclude the ex- for each WMH slice with disease, the closest corresponding traneous tissue outside the prostate in registration evaluation, T2-w MRI slice was visually identified by an expert radiolo- similarity is calculated in terms of the MI ͓Eq. ͑1͔͒ for only gist. the area in the image containing the prostate. Qualitative evaluation is also performed by visually comparing the CaP e extent mapped from histology onto T2-w MRI via TMACMI, V.B.2. Registration experiments e e TPW-T2, and TPW-DCE. An estimate of CaP extent manually As previously described, the goal of this task is to register established by a radiologist on select slices of the T2-w MRI WMH to each MRI protocol in order to map CaP extent onto is used as the ground truth. This is done for only those spe- MRI. Since each MRI series was acquired in sequence and cific MR images where CaP extent could be reliably delin- with minimal movement, 3D affine registration of the DCE eated. It is important to note that these slices are not repre- to T2-w MRI volume was performed for each of the 25 pa- sentative of all T2-w MR images with CaP, since easily c ͑ c͒ tients in S1 and S2 via MI between the 3D T2-w MRI vol- delineated lesions are generally associated with dense tumors ume and the 3D T1-w MRI volume corresponding to the ͑i.e., those with compact cellular arrangements͒50 and those third postcontrast time point of the 4D DCE MRI volume, occurring only in the peripheral zone.51 Further, delineation using 256 gray level bins for the joint histogram. For each of the CaP boundary is more challenging than discerning the axial slice F˜ of the registered 4D DCE MRI volume, a mul- presence of a lesion. tiattribute image representation ␧͑SF˜ 3͒ was generated, as illustrated in Figs. 3͑c͒–3͑e͒. Prior to 2D elastic registration of each H to each ␧͑SF˜ 3͒, the prostate capsule on the corre- VI. RESULTS AND DISCUSSION sponding S images ͑six slices on average͒ for each patient VI.A. Synthetic brain registration was roughly delineated and the extraneous tissue masked out Table II presents a comparison of the evaluation measures during registration. This was done to facilitate a rough global F , F , and D for transformations obtained in elastic localization of the prostate on S relative to H. MAD RMS L2 registration of the n=20 multiprotocol MRI slices using For data set Sc, automatic FFD registration of WMH to 1 MACMI ͑Tco ͒ and both PW registration approaches the MRI by MACMI is then performed for each of the 150 H MACMI ͑Tco and Tco ͒. The values of F were compared be- ␧͑SF˜ 3͒ e͑H͒͒ PW1 PW2 MAD͑ء slices usingI2 ,T , resulting in a warped WMH co co tween TMACMI and TPW1 using a paired t-test under the null H˜ =Te͑H͒, as shown in Fig. 3͑b͒. The CaP extent is then hypothesis that there was no difference in FMAD between S F˜ 3 e ͑ ͒ co co mapped onto and by T , as shown on Fig. 3 c .In TMACMI and TPW1. The values of FRMS and DL2 were also H co co addition to MACMI, the elastic registration of to the co- compared between TMACMI and TPW1. Similarly, the values S ͑ ͒ ordinate frame of is also performed using 1 the conven- of FMAD, FRMS, and DL2 were also compared between H S ͓ ͑S e͑H͔͒͒ ͑ ͒ co co ͑ ͒ tional MI of with I2 ,T and 2 the conven- TMACMI and TPW2 second row of table . MACMI achieves

Medical Physics, Vol. 38, No. 4, April 2011 2014 Chappelow et al.: Multiattribute combined mutual information 2014

TABLE II. Comparison of elastic registration accuracy for MACMI and pair- compared to the left misalignment by PW-T2 in Fig. 5͑f͒. wise MI alignment of n=20 pairs of synthetic PD MRI with coregistered The results in Fig. 5 qualitatively demonstrate that the T1-w and T2-w MRI brain images. The measures illustrated below corre- ͑ ͒ ͑ ͒ general FFD-based registration paradigm described in this spond to i error of recovered deformation field in mm in terms of FMAD ͑ ͒ ͑ ͒ in and FRMS and ii distance DL2 between the undeformed and recovered PD paper is capable of generating good alignment between MRI. MACMI results are significantly more accurate compared to either vivo MRI and ex vivo WMH prostate sections without the use PW approach ͑p-values for both tests shown͒. The best values are indicated of any additional ex vivo MRI series or gross histology in bold. ͑block face͒ photographs. Table III quantitatively illustrates F F D the advantage of using MACMI over a single MRI protocol. MAD RMS L2 c For each of the n=25 patient studies in S1, registration accu- co ͑ ͒ ϫ +03 TPW1 T1-PD 0.9117 2.1407 1.83 10 racy for each of MACMI, PW-T2, and PW-DCE was ap- co ͑ ͒ ϫ +03 TPW2 T2-PD 0.9506 2.0248 2.35 10 proximated by the total MI of the elastically registered his- Tco ͓␧͑T1T2͒-PD͔ 0.8348 1.9307 1.71ϫ10+03 MACMI tology slices H˜ with all of the corresponding ͑1͒ S and ͑2͒ p ͑Tco vs Tco ͒ 0.0817 0.0578 0.0174 PW1 MACMI 3 ͑ co co ͒ ϫ −10 F˜ slices, both of which are in the same coordinate frame as p TPW2 vs TMACMI 0.0013 0.2020 1.8 10 S ¯ . Table III lists the average MI value I2 over n=25 patients of all registered WMH slices H˜ obtained by MACMI, PW- ͑ ͒ S F˜ 3 ͑ ͒ better performance in terms of each measure, with signifi- T2, and PW-DCE columns with either or rows . The ¯ ͑S H˜ ͓͒ ¯ ͑F˜ 3 H˜ ͔͒ cantly lower error ͑pϽ0.05 for n=20͒ compared to one or values of I2 , and I2 , were compared between both PW methods. MACMI and PW-T2 using a paired t-test under the null hy- ¯ ͑S H˜ ͓͒ pothesis that there was no difference in I2 , and ¯ ͑F˜ 3 H˜ ͔͒ VI.B. Clinical prostate registration I2 , between MACMI and PW-T2. The comparisons ¯ ͑S H˜ ͒ ¯ ͑F˜ 3 H˜ ͒ VI.B.1. Mapping CaP extent from WMH onto in vivo of I2 , and I2 , were also made between MACMI T2-w and T1-w MRI and PW-DCE. In all comparisons, MACMI demonstrated significant ͑pϽ0.05͒ improvement over both PW-T2 and Figure 5 illustrates the registered WMH images and the PW-DCE, despite these PW methods using MI as their ob- corresponding T2-w MRI along with the contours of the jective function. mapped CaP extent and the urethra ͑verumontanum͒, for a single set of corresponding images in Sc ͑and Sc͒. The S slice 1 2 VI.B.2. Mapping CaP extent from WMH onto ADC, is shown in Fig. 5͑a͒ with the region containing the dominant T2-w and T1-w MRI intraprostatic lesion ͑DIL͒ shown in the box and the urethra c outlined. An expert delineation of the DIL is shown in Fig. For the m=15 patient studies in S2 for which ADC maps 5͑b͒, while the original WMH image with CaP ground truth were also obtained, MACMI was applied in both ͑1͒ the 3D ͑dotted line͒ is shown in Fig. 5͑c͒. As described in Sec. IV B, affine alignment of D to ␧͑SF˜ 3͒ and ͑2͒ the 2D elastic align- affine registration of T2-w and DCE MRI protocol volumes ment of H to ␧͑SF˜ 3D˜ ͒. Figures 6͑a͒ and 6͑b͒ show the origi- ͑containing images S with F͒ is performed prior to elastic nal H and the warped H ͓Te͑H͔͒, following elastic align- H H ͓ registration of . Elastic registration of the slice shown ment with ␧͑SF˜ 3D˜ ͒. Figure 6͑c͒ shows a checker board in Fig. 5͑c͔͒ to S in Fig. 5͑a͒ is then performed individually visualization of the two coregistered S and F˜ 3 slices ͓shown using each method ͑PW-T2, PW-DCE, and MACMI͒. The in Figs. 6͑d͒ and 6͑e͔͒ after the 3D registration of the T2-w H ͑H˜ ͒ S ͑ ͒ elastically warped , obtained using only PW-T2 ,is and DCE MRI volumes performed for the studies in Sc. For ͑ ͒ 1 shown in Fig. 5 f and the CaP extent is shown mapped onto each of the studies in Sc ʚSc, the second 3D affine multipro- ˜ 2 1 S in Figs. 5͑d͒ and 5͑e͒. H obtained using only F ͑PW-DCE͒ tocol registration step was performed to align the ADC vol- is shown in Fig. 5͑j͒ and the CaP extent is again shown ume to the T2-w and DCE MRI volumes. This enables the ˜ mapped onto S ͓Figs. 5͑h͒ and 5͑i͔͒. Finally, H obtained generation of aligned D˜ images ͓Fig. 6͑f͔͒ for each S ͑and S F ͑ ͒ ͑ ͒ using both and MACMI is shown in Fig. 5 m and the F˜ 3͒. S and the registered F˜ 3 and D˜ images are shown in S ͓ ͑ ͒ ͑ ͔͒ CaP extent is again shown on Figs. 5 k and 5 l . The Figs. 6͑d͒–6͑f͒ with the contour of the mapped CaP extent ͑ ͒ contours of the mapped CaP extent on MRI suggest that 1 from Te͑H͓͒Fig. 6͑b͔͒. It was observed that the inclusion of accurate elastic registration of WMH directly to correspond- D˜ in ␧͑SF˜ 3D˜ ͒ had little effect on the resulting alignment ing in vivo MRI is feasible using the described FFD frame- with H when compared with the results obtained using only work and ͑2͒ MACMI outperforms PW application of MI S F˜ 3 ͑ ͒ using data from only one of the several available MRI pro- and as in Sec. VI B 1 above comparison not shown . tocols. Note again that while the DIL shown in Fig. 5 is useful for qualitative evaluation, identifying such a clearly VII. CONCLUDING REMARKS bounded lesion on T2-w MRI is rare. The positions of the Signatures for disease on multimodal in vivo imaging may urethra on H˜ in Figs. 5͑f͒, 5͑j͒, and 5͑m͒ also illustrate im- be used to develop systems for computer-assisted detection proved alignment of the image interior via MACMI com- of cancer or to assist in the training of medical students, pared to PW-T2 and PW-DCE. For example, note the im- radiology residents, and fellows. To establish in vivo radio- proved urethral positioning via MACMI in Fig. 5͑m͒ logical imaging signatures for prostate cancer, an accurate

Medical Physics, Vol. 38, No. 4, April 2011 2015 Chappelow et al.: Multiattribute combined mutual information 2015

(a) (b) (c) 2 PW-T

(d) (e) (f) PW-DCE

(h) (i) (j) MACMI

(k) (l) (m) Under-segmentation Over-segmentation

FIG.5. ͑a͒ 3T in vivo T2-w MRI of a prostate with a clearly visible DIL and magniﬁed in ͑b͒ with a manual estimate of CaP extent. ͑c͒ Closest corresponding WMH slice with CaP ground truth ͑dotted line͒ and urethra. ͓͑d͒ and ͑e͔͒ T2-w MRI with estimate of CaP extent as mapped from ͑f͒ WMH via elastic registration using only T2-w MRI. ͓͑h͒ and ͑i͔͒ T2-w MRI with CaP estimate from ͑j͒ WMH registered to DCE ͑T1-w͒ MRI ͑coregistered to T2-w MRI͒. ͓͑k͒ and ͑l͔͒ Registration using both T2-w and DCE MRI via MACMI results in closer agreement of the registration-derived CaP extent and the manual estimate. The verumontanum of the urethra is also shown on the registered WMH images in ͑f͒, ͑j͒, and ͑m͒. estimate of ground truth for cancer extent on each of the tology sections with corresponding in vivo images. In this imaging modalities is necessary. In the context of certain paper, we presented a new method termed MACMI within an anatomic regions and diseases, the spatial extent of disease automated elastic FFD registration framework for alignment may be obtained by spatial correlation or registration of his- of images from multiple in vivo acquisition protocols with

Medical Physics, Vol. 38, No. 4, April 2011 2016 Chappelow et al.: Multiattribute combined mutual information 2016

TABLE III. Comparison of elastic registration accuracy for MACMI, PW-T2, and PW-DCE for n=25 patient studies. The measures illustrated below correspond to mean similarity in terms of the total MI of all registered WMH slices H˜ , obtained by MACMI, PW-T2, or PW-DCE ͑columns of table͒, with either T2-w MRI or DCE MRI ͑rows of table͒.

Mean MI with WMH Registration objective function p ͑vs MACMI͒

MR protocol MACMI PW-T2 PW-DCE PW-T2 PW-DCE

¯ ͑S H˜ ͒ T2-w: I2 , 0.3378 0.3339 0.3297 0.0099 0.0006 ¯ ͑F˜ 3 H˜ ͒ ϫ −4 DCE: I2 , 0.3155 0.3085 0.3102 4.0 10 0.0014

corresponding ex vivo WMH sections. Our approach to reg- highly dissimilar modalities and large deformations of vari- istration of in vivo multiprotocol radiology images and ex able magnitude. vivo WMH of the prostate using MACMI is distinct from We demonstrated the use of MACMI for registration of previous related efforts18,30,35 in that ͑1͒ information from all 150 multimodal ͑WMH, T2-w, and DCE MRI͒ prostate im- in vivo image sources is being utilized simultaneously to age sets from 25 patients; 85 sets of WMH, T2-w, ADC, and drive the automated elastic registration with WMH; ͑2͒ no DCE MRI from 15 patients; and 20 sets of synthetic T1-w, additional, intermediate ex vivo radiology or gross histology T2-w, and PD MR brain images. Statistically significant im- images need to be obtained ͑this approach does not disrupt provement in registration accuracy was observed in using routine clinical workflow͒; and ͑3͒ no point correspondences MACMI to simultaneously register PD MRI to both T1-w are required to be identified manually or automatically. This and T2-w MRI, compared to pairwise registration of PD to last advantage is particularly relevant in the context of in T1-w or T2-w MRI, for the synthetic data set. Qualitative vivo MR images where visual identification of anatomical examination of alignment between multiprotocol clinical landmarks is a challenge even for experts. prostate MRI and histology suggested improved performance MACMI performs registration of several images by incor- via MACMI over pairwise MI. The inclusion of ADC MRI porating multiple image sources using an information theo- in the multiattribute registration had little effect on the result- retic approach. These may include different modalities, ac- ing alignment with WMH when compared to the results ob- quisition protocols, or image features. For the clinical tained using only T2-w and T1-w MRI ͑Sec. VI B 1͒. Nev- application discussed in this paper, MACMI facilitates the ertheless, it is possible that the use of MACMI in the use of all available in vivo prostate imaging protocols ac- registration of ADC to the ensemble of registered T2-w and quired during the standard clinical routine in order to per- T1-w MRI helped achieve more consistent multiprotocol form automated elastic registration with the ex vivo WMH. MRI alignment than if either protocol were used alone. We Unlike fully groupwise registration techniques, the optimiza- intend to investigate this application of MACMI further in tion problem remains simple while accommodating both the future. While we utilized histograms for density estimation, other techniques, such as entropic graphs,44 can be applied for larger numbers of images. However, independent of the implemented estimation method, MACMI affects information theoretic fusion of multiple image sources by computing multivariate MI between multiattribute images constructed as ensembles of coregistered images. It is important to note that in the absence of a predetermined order for combining (a) (b) (c) images, MACMI may still be applied by combining images in a completely arbitrary order. Even in this scenario, MACMI still represents an improvement over fully pairwise registration by utilizing all registered images. Future work will investigate the influence of the order of multiattribute image construction on alignment accuracy.

(d) (e) (f) ACKNOWLEDGMENTS FIG. 6. Using MACMI to include ADC MRI in the elastic registration of ͑a͒ histology to each of ͑d͒ T2-w, ͑e͒ DCE ͑T1-w͒, and ͑f͒ ADC MRI. Prior to This work was made possible via grants from the Depart- elastic registration of histology, ͓͑d͒ and ͑e͔͒ T2-w and T1-w MRI were ﬁrst ment of Defense Prostate Cancer Research Program ͑Grant successfully aligned via MI, as seen by ͑c͒ the checkerboard overlay of No. W81XWH-08-1-0072͒, Wallace H. Coulter Foundation, T2-w MRI and registered T1-MRI. ͑e͒ ADC was then registered to both ͑ ͑ ͒ National Cancer Institute Grant Nos. R01CA136535-01, T2-w and T1-w MRI via MACMI. b Elastically registered histology was ͒ obtained using the coregistered multiprotocol MRI via MACMI and CaP R01CA140772-01, and R03CA143991-01 , and The Cancer extent was mapped onto ͓͑d͒–͑f͔͒ MRI. Institute of New Jersey.

Medical Physics, Vol. 38, No. 4, April 2011 2017 Chappelow et al.: Multiattribute combined mutual information 2017

͒ a Author to whom correspondence should be addressed. Electronic mail: 1039 ͑2008͒. [email protected]; URL: http://lcib.rutgers.edu/ 19Y. Ou, D. Shen, M. Feldman, J. Tomaszewski, and C. Davatzikos, “Non- 1B. N. Bloch, E. Furman-Haran, T. H. Helbich, R. E. Lenkinski, H. De- rigid registration between histological and MR images of the prostate: A gani, C. Kratzik, M. Susani, A. Haitel, S. Jaromi, L. Ngo, and N. M. joint segmentation and registration framework,” in Computer Vision and Rofsky, “Prostate cancer: Accurate determination of extracapsular exten- Pattern Recognition Workshops ͑IEEE Computer Society, Los Alamitos, sion with high-spatial-resolution dynamic contrast-enhanced and T2- 2009͒ pp. 125–132. weighted MR imaging–initial results,” Radiology 245, 176–185 ͑2007͒. 20J. Pluim, J. Maintz, and M. A. Viergever, “Image registration by maxi- 2J. Kurhanewicz, D. Vigneron, P. Carroll, and F. Coakley, “Multiparamet- mization of combined mutual information and gradient information,” ric magnetic resonance imaging in prostate cancer: Present and future,” IEEE Trans. Med. Imaging 19, 809–814 ͑2000͒. Current Opinion Urology 18, 71–77 ͑2008͒. 21D. Rueckert, M. Clarckson, and D. J. Hawkes, “Non-rigid registration 3C. M. A. Hoeks, J. J. Fütterer, D. M. Somford, I. M. van Oort, H. Huis- using higher-order mutual information,” Proc. SPIE 3979, 438–447 man, and J. O. Barentsz, “Multiparametric MRI for prostate cancer ͑2000͒. screening,” Ned Tijdschr Geneeskd 153, B487-1–B487-7 ͑2009͒. 22J. Boes and C. Meyer, “Multi-variate mutual information for registra- 4D. L. Langer, T. H. van der Kwast, A. J. Evans, J. Trachtenberg, B. C. tion,” in Proceedings of MICCAI, Vol. 1679, pp. 606–612, 1999. Wilson, and M. A. Haider, “Prostate cancer detection with multi- 23C. Studholme, D. Hill, and D. J. Hawkes, “Incorporating connected re- parametric MRI: Logistic regression analysis of quantitative T2, gion labeling into automatic image registration using mutual informa- diffusion-weighted imaging, and dynamic contrast-enhanced MRI,” J. tion,” in Mathematical Methods in Biomedical Image Analysis, Vol. 3979, Magn. Reson Imaging 30, 327–334 ͑2009͒. pp. 23–31, 1996. 5S. Viswanath, B. N. Bloch, M. Rosen, J. Chappelow, and R. Toth, “Inte- 24B. N. Bloch, R. E. Lenkinski, and N. M. Rofsky, “The role of magnetic grating structural and functional imaging for computer assisted detection resonance imaging ͑MRI͒ in prostate cancer imaging and staging at 1.5 of prostate cancer on multi-protocol in vivo 3 Tesla MRI,” Proc. SPIE and 3 Tesla: The Beth Israel Deaconess Medical Center ͑BIDMC͒ ap- 7260, 72603I-1–72603I-12 ͑2009͒. proach,” Cancer Epidemiology Biomarkers and Prevention 4, 251–262 6P. Tiwari, M. Rosen, and A. Madabhushi, “A hierarchical spectral clus- ͑2008͒. tering and nonlinear dimensionality reduction scheme for detection of 25J. Kurhanewicz, M. G. Swanson, S. J. Nelson, and D. B. Vigneron, prostate cancer from magnetic resonance spectroscopy ͑MRS͒,” Med. “Combined magnetic resonance imaging and spectroscopic imaging ap- Phys. 36, 3927–3939 ͑2009͒. proach to molecular imaging of prostate cancer,” J. Magn. Reson Imaging 7M. A. Haider, T. H. van der Kwast, J. Tanguay, A. J. Evans, A.-T. 16, 451–463 ͑2002͒. Hashmi, G. Lockwood, and J. Trachtenberg, “Combined T2-weighted and 26A. Bharatha et al., “Evaluation of three-dimensional finite element-based diffusion-weighted MRI for localization of prostate cancer,” AJR, Am. J. deformable registration of pre- and intraoperative prostate imaging,” Roentgenol. 189, 323–328 ͑2007͒. Med. Phys. 28, 2551–2560 ͑2001͒. 8P. Kozlowski, S. D. Chang, E. C. Jones, K. W. Berean, H. Chen, and S. L. 27M. Foskey, B. Davis, L. Goyal, S. Chang, E. Chaney, N. Strehl, S. Tomei, Goldenberg, “Combined diffusion-weighted and dynamic contrast- J. Rosenman, and S. Joshi, “Large deformation three-dimensional image enhanced MRI for prostate cancer diagnosis—Correlation with biopsy registration in image-guided radiation therapy,” Phys. Med. Biol. 50, and histopathology,” J. Magn. Reson Imaging 24, 108–113 ͑2006͒. 5869–5892 ͑2005͒. 9A. Madabhushi, M. D. Feldman, D. N. Metaxas, J. Tomaszeweski, and D. 28S. Xu, J. Kruecker, B. Turkbey, N. Glossop, A. K. Singh, P. Choyke, P. Chute, “Automated detection of prostatic adenocarcinoma from high- Pinto, and B. J. Wood, “Real-time MRI-TRUS fusion for guidance of resolution ex vivo MRI,” IEEE Trans. Med. Imaging 24, 1611–1625 targeted prostate biopsies,” Comput. Aided Surg. 13, 255–264 ͑2008͒. ͑2005͒. 29A. K. Singh, J. Kruecker, S. Xu, N. Glossop, P. Guion, K. Ullman, P. L. 10P. Tiwari, J. Kurhanewicz, M. Rosen, and A. Madabhushi, “Semi super- Choyke, and B. J. Wood, “Initial clinical experience with real-time tran- vised multi kernel ͑SeSMiK͒ graph embedding: Identifying aggressive srectal ultrasonography-magnetic resonance imaging fusion-guided pros- prostate cancer via magnetic resonance imaging and spectroscopy,” in tate biopsy,” BJU Int. 101͑5͒, 841–845 ͑2008͒. Proceedings of MICCAI, Vol. 6363, 666–673 ͑2010͒. 30Z. Lee, D. B. Sodee, M. Resnick, and G. T. Maclennan, “Multimodal and 11J. Chappelow, S. Viswanath, J. Monaco, M. Rosen, J. Tomaszewski, M. 3D imaging of prostate cancer,” Comput. Med. Imaging Graph. 29, 477– Feldman, and A. Madabhushi, “Improving supervised classification accu- 486 ͑2005͒. racy using non-rigid multimodal image registration: Detecting prostate 31C. C. Parker, A. Damyanovich, T. Haycocks, M. Haider, A. Bayley, and cancer,” Proc. SPIE 6915, 69150V-1–69150V-12 ͑2008͒. C. N. Catton, “Magnetic resonance imaging in the radiation treatment 12C. Bartolozzi, I. Menchi, R. Lencioni, S. Serni, A. Lapini, G. Barbanti, A. planning of localized prostate cancer using intra-prostatic fiducial markers Bozza, A. Amorosi, A. Manganelli, and M. Carini, “Local staging of for computed tomography co-registration,” Radiother. Oncol. 66, 217– prostate carcinoma with endorectal coil MRI: Correlation with whole- 224 ͑2003͒. mount radical prostatectomy specimens,” Eur. Radiol. 6, 339–345 ͑1996͒. 32H. J. Huisman, J. J. Fütterer, E. N. J. T. van Lin, A. Welmers, T. W. J. 13P. C. Vos, T. Hambrock, C. A. H. van de Kaa, J. J. Fütterer, J. O. Bar- Scheenen, J. A. van Dalen, A. G. Visser, J. A. Witjes, and J. O. Barentsz, entsz, and H. J. Huisman, “Computerized analysis of prostate lesions in “Prostate cancer: Precision of integrating functional MR imaging with the peripheral zone using dynamic contrast enhanced MRI,” Med. Phys. radiation therapy treatment by using fiducial gold markers,” Radiology 35, 888–899 ͑2008͒. 236, 311–317 ͑2005͒. 14S. Ozer, D. L. Langer, X. Liu, M. A. Haider, T. H. van der Kwast, A. J. 33S. Vidakovic, H. S. Jans, A. Alexander, and R. S. Sloboda, “Post-implant Evans, Y. Yang, M. N. Wernick, and I. S. Yetik, “Supervised and unsu- computed tomography—Magnetic resonance prostate image registration pervised methods for prostate cancer segmentation with multispectral using feature line parallelization and normalized mutual information,” J. MRI,” Med. Phys. 37, 1873–1883 ͑2010͒. Appl. Clin. Med. Phys. 8͑1͒, 21–32 ͑2007͒. 15X. Liu, D. L. Langer, M. A. Haider, Y. Yang, M. N. Wernick, and I. S. 34H. Park, S. Kwee, G. Thibault, R. Stack, I. Sesterhenn, K. Potter, and C. Yetik, “Prostate cancer segmentation with simultaneous estimation of Meyer, “Registration methods for histological slides and ex vivo MRI of Markov random field parameters and class,” IEEE Trans. Med. Imaging prostate,” in IEEE Nuclear Science Symposium Conference Record, 28, 906–915 ͑2009͒. 2007, Vol. 4, pp. 3102–3104, 2007. 16J. Chappelow and A. Madabhushi, “Multi-attribute combined mutual in- 35C. R. Meyer, B. A. Moffat, K. K. Kuszpit, P. L. Bland, T. L. Chenevert, formation ͑MACMI͒: An image registration framework for leveraging A. Rehemtulla, and B. D. Ross, “A methodology for registration of a multiple data channels,” in Proceedings of the IEEE International Sym- histological slide and in vivo MRI volume based on optimizing mutual posium on Biomedical Imaging, 2010. information,” Mol. Imaging 5,16–23͑2006͒. 17P. C. Vos, T. Hambrock, J. O. Barenstz, and H. J. Huisman, “Computer- 36S. B. Park, F. C. Rhee, J. I. Monroe, and J. W. Sohn, “Spatially weighted assisted analysis of peripheral zone prostate lesions using T2-weighted mutual information image registration for image guided radiation and dynamic contrast enhanced T1-weighted MRI,” Phys. Med. Biol. 55, therapy,” Med. Phys. 37, 4590–4601 ͑2010͒. 1719–1734 ͑2010͒. 37Y. Zhan, Y. Ou, M. Feldman, J. Tomaszeweski, and C. Davatzikos, “Reg- 18H. Park, M. R. Piert, A. Khan, R. Shah, H. Hussain, J. Siddiqui, T. L. istering histologic and MR images of prostate for image-based cancer Chenevert, and C. R. Meyer, “Registration methodology for histological detection,” Acad. Radiol. 14, 1367–1381 ͑2007͒. sections and in vivo imaging of human prostate,” Acad. Radiol. 15, 1027– 38J. Chappelow, B. N. Bloch, N. Rofsky, E. Genega, R. Lenkinski, W.

Medical Physics, Vol. 38, No. 4, April 2011 2018 Chappelow et al.: Multiattribute combined mutual information 2018

DeWolf, S. Viswanath, and A. Madabhushi, “COLLINARUS: Collection 46D. L. Collins, A. P. Zijdenbos, V. Kollokian, J. G. Sled, N. J. Kabani, C. of image-derived non-linear attributes for registration using splines,” J. Holmes, and A. C. Evans, “Design and construction of a realistic digital Proc. SPIE 7259, 72592N-1–72592N-12 ͑2009͒. brain phantom,” IEEE Trans. Med. Imaging 17, 463–468 ͑1998͒. 39K. K. Bhatia, J. V. Hajnal, B. K. Puri, A. D. Edwards, and D. Rueckert, 47J. C. Lagarias, J. A. Reeds, M. H. Wright, and P. E. Wright, “Convergence “Consistent groupwise non-rigid registration for atlas construction,” in properties of the Nelder–Mead simplex method in low dimensions,” Proceedings of the IEEE International Symposium on Biomedical Imag- SIAM J. Optim. 9, 112–147 ͑1998͒. ing, pp. 908–911, 2004. 48G. Xiao, B. N. Bloch, J. Chappelow, E. M. Genega, N. M. Rofsky, R. E. 40C. Studholme, “Simultaneous population based image alignment for tem- Lenkinski, J. Tomaszewski, M. D. Feldman, M. Rosen, and A. Madab- plate free spatial normalisation of brain anatomy,” in Workshop on Bio- hushi, “Determining histology-MRI slice correspondences for deﬁning medical Imaging Registration, pp. 81–90, 2003. MRI-based disease signatures of prostate cancer,” Comput. Med. Imaging 41T. F. Cootes, S. Marsland, C. Twining, K. Smith, and C. Taylor, “Group- Graph. ͑in press͒. wise diffeomorphic non-rigid registration for automatic model building,” 49J. Chappelow, J. E. Tomaszewski, M. Feldman, N. Shih, and A. Madab- in European Conference on Computer Vision, pp. 316–327, 2004. hushi, “Histostitcher: An interactive program for accurate and rapid re- 42S. Balci, P. Golland, M. Shenton, and W. Wells, “Free-form B-spline construction of digitized whole histological sections from tissue frag- deformation model for groupwise registration,” in Proceedings of ments,” Comput. Med. Imaging Graph. ͑2011͒͑in press͒. MICCAI, 2007. 50V. Nicolas, D. Beyersdorff, U. Mueller-Lise, W. Pennekamp, and C. 43http://www.bic.mni.mcgill.ca/brainweb/ Heyer, “The prostate and seminal vesicles,” in MR Imaging of the Abdo- 44H. Neemuchwala, A. Hero, and P. Carson, “Image matching using alpha- men and Pelvis ͑Thieme, 2002͒, pp. 220–240. entropy measures and entropic graphs,” Signal Process. 85, 277–296 51M. L. Schiebler, J. E. Tomaszewski, M. Bezzi, H. M. Pollack, H. Y. ͑2005͒. Kressel, E. K. Cohen, H. G. Altman, W. B. Gefter, A. J. Wein, and L. 45M. Staring, U. A. van der Heide, S. Klein, M. A. Viergever, and J. Pluim, Axel, “Prostatic carcinoma and benign prostatic hyperplasia: Correlation “Registration of cervical MRI using multifeature mutual information,” of high-resolution MR and histopathologic ﬁndings,” Radiology 172, IEEE Trans. Med. Imaging 28, 1412–1421 ͑2009͒. 131–137 ͑1989͒.

Medical Physics, Vol. 38, No. 4, April 2011