2/2/2012 Medical Imaging Analysis Image Registration in Medical Imaging y Developing mathematical algorihms to extract and relate information from medical images y Two related aspects of research BI260 { Image Registration VALERIE CARDENAS NICOLSON, PH.D Ù Fin ding spatia l/tempora l correspondences between image data and/or models ACKNOWLEDGEMENTS: { Image segmentation Ù Extracting/detecting specific features of interest from image data COLIN STUDHOLME, PH.D. LUCAS AND KANADE, PROC IMAGE UNDERSTANDING WORKSHOP, 1981 Medical Imaging Regsitration: Overview Registration y What is registration? { Definition “the determination of a one-to-one mapping between the coordinates in one space and those in { Classifications: Geometry, Transformations, Measures another, such that points in the two spaces that y Motivation for work correspond to the same anatomical point are mapped to each other.” { Where is image regiistrat ion used in medic ine and biome dica l research? Calvin Maurer ‘93 y Measures of image alignment { Landmark/feature based methods { Voxel-based methods Ù Image intensity difference and correlation Ù Multi-modality measures 1 2/2/2012 Image Registration Key Applications I: Change detection y Look for differences in the same type of images taken at different times, e.g.: { Mapping pre and post contrast agent “Establishing correspondence, Ù Digital subtraction angiography (DSA) between features Ù MRI with gadolinium tracer in sets of images, and { Mapping structural changes using a transformation model to Ù Different stages in tumor growth (before and after treatment) infer correspondence Ù Neurodegenerative disease-> qqyguantifying tissue loss p atterns away from those features.” { Detecting changes due to function Ù Functional MRI (fMRI): before and after brain stimulus Bill Crum ‘05 Ù PET imaging: quantitative tracer uptake measurements y Problem: { Subject scanned multiple times-> removed from scanner { We cannot easily fix/know patient location and orientation with respect to imaging system { Need to remove differences in patient positioning to detect true differences in patient images Key Applications II: Image Fusion Components of Image Registration Algorithms y Relate contrasting information from different types of y Image Data Geometries images { 2D-2D, 2D-3D, 3D-3D y Multi-modality imaging y Transformation type { MRI-CT { Rigid/affine/non-rigid { MRI-PET/SPECT { Structural MRI-functional MRI y Correspondence criteria/measure { Structural MRI to structural MRI { Feature based methods y Problem: { voxel based/dense field methods { Subject scanned multiple times-> different scanners { We cannot easily fix/know patient location and orientation with y Optimization method respect to different imaging systems { Maximizing/minimizing criteria with respect to transform { Need to remove differences in patient positioning to relate information from different types of images 2 2/2/2012 2D-2D Inter-modality image registration problem Registration and display of the combined bone scan and radiograph in the Examples of Image Geometries and diagnosis and management of write injuries, Hawkes et al., EJNM 1991 Transformation Models in Medical Applications 2D-2D image transformations 2D-2D image transformations y Simple case: parallel projection { 2 translations t1 and t2 (up-down/left-right) and one rotation (θ) This is a linear mapping from (x1,x2) to (y1, y2) y1=x1cos θ-x2sin θ+t1 y2=x1sin θ+x2cos θ+t2 3 2/2/2012 Rotating around a given location Image Scaling y Scaling with respect to a fixed point (x,y) y Scaling along an arbitrary axis Influence of Affine Transformations on Images Composing Transformations y Map lines to lines y Map parallel lines to parallel lines y Preserve ratios of distances along a line y Do NOT ppgreserve absolute distances and angles 4 2/2/2012 2D-3D registration problems 2D-3D Registration Geometry y Radiotherapy “the determination of a projection mapping, { Treatment verification (Portal images) from a 3D to a 2D coordinate system such that points in each space which correspond to the { Treatment planning (simulator system) same anatomical points are mapped to each { Treatment guidance (Cyberknife system) other.” y Orthopaedic surgery { Spinal surgery (Brain Lab, Medtronic) { Hip or Knee arthroplasty (ROBODOC) Ù Verification of implant position y Neurointerventions { Matching MRA to DSA y Surgical microscope { MRI/CT neurosurgery guidance y Virtual endoscopy (A) Imaging/Acquisition parameters (intrinsic) (B) Subject Parameters (extrinsic) 5 2/2/2012 3D-3D image registration 3D Rigid Transformations ⎡x'⎤ ⎡t11 t12 t13 t14 ⎤⎡x⎤ y Many different 3D clinical imaging modalities ⎢y'⎥ ⎢t t t t ⎥⎢y⎥ ⎢ ⎥ = ⎢ 21 22 23 24 ⎥⎢ ⎥ { MRI probably still the least common ⎢z'⎥ ⎢t31 t32 t33 t34 ⎥⎢z⎥ ⎢ ⎥ ⎢ ⎥⎢ ⎥ ⎣1 ⎦ ⎣t41 t42 t43 t44 ⎦⎣1⎦ y Images used in many different clinical settings ⎡1 0 0 0⎤ ⎡cos β 0 − sin β 0⎤ ⎡ cosθ sinθ 0 0⎤ { Diagnosis ⎢0 cosα sinα 0⎥ ⎢ 0 1 0 0⎥ ⎢− sinθ cosθ 0 0⎥ R = ⎢ ⎥ R = ⎢ ⎥ R = ⎢ ⎥ { Treatment planning x ⎢0 − sinα cosα 0⎥ y ⎢sin β 0 cos β 0⎥ z ⎢ 0 0 1 0⎥ { Treatment guidance ⎢ ⎥ ⎢ ⎥ ⎢ ⎥ ⎣0 0 0 1⎦ ⎣ 0 0 0 1⎦ ⎣ 0 0 0 1⎦ { Clinical research: studying disease z θ ⎡1 0 0 x0 ⎤ y Transformation types: ⎢ ⎥ 0 1 0 y1 { Rigid positioning of subject: still most common T = ⎢ ⎥ ⎢0 0 1 z1 ⎥ { Non-rigid deformations to describe y ⎢ ⎥ Ù Tissue deformation α ⎣0 0 0 1 ⎦ Ù Imaging distortion β Ù Differences between subjects x Feature Based Approaches y Point set -> Point set (homologous) y Point set -> Point set (non homologous… so need to find order) y Point set -> Surface y Surface -> Surface (also Curve->Surface) 6 2/2/2012 MR-CT Registration Homologous Feature Based Alignment y Manual point landmark y General case of two lists of identification (around 12 corresponding feature locations points) in MR and CT { [p1, p2,…,pN] and { [q1, q2,…,qN] both with N points y Relates soft tissue structures such as y We want to find: enhancing tumor and TfTransforma tion T(q) tha t minimizes squared distance blood flow to bone between corresponding points: features in CT 2 E = ∑ pr −T (qr ) r y Where one set of points, q, is transformed by T() -> Extrapolate Transformation for all image voxels/pixels Procrustes Point Alignment Golub and VanLoan, Matrix Computations, 1989 Procrustes Point Alignment y Remove translational differences y Scale (procrustes normally includes estimate of scaling) { Calculate translation that brings each of the point sets to { But we can assume scanner voxel dimensions are accurate the origin and subtract from each of the point sets to create centered point sets 1 y It is possible to show that the rotation solution is: q'i = qi − ∑qi N i 1 p'i = pi − ∑ pi T N i R=VU Rewrite centered point lists as matrices ⎡ p'1 ⎤ ⎡x1 y1 z1 ⎤ ⎡ q'1 ⎤ ⎡x'1 y'1 z'1 ⎤ where U and V are matrices obtained ⎢ p' ⎥ ⎢x y z ⎥ ⎢q' ⎥ ⎢x' y' z' ⎥ T P = ⎢ 2 ⎥ = ⎢ 2 2 2 ⎥ Q = ⎢ 2 ⎥ = ⎢ 2 2 2 ⎥ from the SVD of P Q ⎢ M ⎥ ⎢ M M M ⎥ ⎢ M ⎥ ⎢ M M M ⎥ ⎢ ⎥ ⎢ ⎥ ⎢ ⎥ ⎢ ⎥ p' x y z q' x' y' z' ⎣ N ⎦ ⎣ 3 3 3 ⎦ ⎣ N ⎦ ⎣ 3 3 3 ⎦ Recall that (PTQ->(USVT) { Estimate rotations: find the rotation matrix R such that PT=RQT S is a diagonal matrix, U and V are matrices K.S. Arun, T.S. Huang, and S.D. Blostein, Least square fitting of two 3-d point sets. IEEE PAMI, containing the left and right singular vectors. T T The system P =RQ is over-determined and there is 9(5):698-700, 1987. noise, thus we want to find the rotation matrix For 2D: ⎡cosθ − sinθ ⎤ R such that VU T = T T 2 ⎢ ⎥ minR=|| P -RQ || ⎣sinθ cosθ ⎦ 7 2/2/2012 Extrapolating Transformations: Theoretical point based registration error Alternatives to SVD alignment (points on circle 50 mm radius selected with RMS error of 2mm) y Quaternion methods { Horn, Closed-form solution of absolute orientation using unit quaternions. Journal of the Optical Society of America A, 4(4):629-642, 1987. y Orthonormal matrices { Horn et al., Closed-form solution of absolute orientation using orthonormal matrices. Journal of the Optical Society of America A, 5(7):1127-1135, July 1988. y Dual quaternions { Walker et al., Estimating 3-d location parameters using dual number quaternions. CVGIP: Image Understanding, 54:358-367, November 1991. y These algorithms generally show similar performance and stability with real world noisy data { Loruss et al., A comparison of four algorithms for estimating 3-D rigid transformations. In Proceedings of the 4th British machine conference (BVMC ‘95), pages 237-246, Birmingham, England, September 1995. Important factor: registration lowest where 3D landmarks are found The distribution of target registration error in rigid-body point-based registration. Fitzpatrick and West, IEEE TMI, 20(9): 2001, 917-927. Approaches to Landmark/Feature Extraction and Matching y Markers attached to subject Early feature based brain/head matching { Need to be visible in different types of images { Hawkes et al., Registration and display of the combined bone scan and radiograph in the diagnosis and management of wrist injuries, EJNM 1991. y Manual landmark identification { Time consuming, introduce errors, difficult to find true consistent 3D landmarks, but VERY flexible and can adapt to data { Hill et al., Accurate frameless registration of MR and CT images of the head: Applications in surgery and radiotherapy planning, Radiology 1994, 447-454. y Automated landmark identification: geometric models of local anatomy { Need to be true unique 3D point in intensity map: tip-like, saddle-like, and sphere like structures { Need to be consistent in different subjects and image types { Worz and Rohr, Localization of anatomical point landmarks in 3D medical images by fitting 3D parametric intensity models, Medical Image Analysis 10(1):41-58, 2006. y Non-homologous landmarks/3D structures { Easier to automatically find general features: for example point on a surface using edge detection { But: which point maps to which point? “Head-hat” matching of head surfaces { Need to then find correspondence and alignment: point cloud fitting Retrospective geometric correlation of MR, CT and PET images, Levin et al., Radiology 1988.