9/7/17 Visual Computing in Medicine Hans-Christian Hege Int. Summer School 2017 on Deep Learning and Visual Data Analysis, Ostrava, 07.Sept.2017 Acknowledgements Hans Lamecker Stefan Zachow Dagmar Kainmüller Heiko Ramm Britta Weber Daniel Baum ’ 1 9/7/17 Visual Computing Image-Related Disciplines Data Processing Non-Visual Data Image/Video Analysis Data Acquisition Computer Graphics Computer Vision Computer Animation Imaging VR, AR Data Visualization Visual Data Image/Video Processing 2 9/7/17 Visual Computing source: Wikipedia Visual computing = all computer science disciplines handling images and 3D models, i.e. computer graphics, image processing, visualization, computer vision, virtual and augmented reality, video processing, but also includes aspects of pattern recognition, human computer interaction, machine learning and digital libraries. Core challenges are the acquisition, processing, analysis and rendering of visual information (mainly images and video). Application areas include industrial quality control, medical image processing and visualization, surveying, robotics, multimedia systems, virtual heritage, special effects in movies and television, and computer games. Images (mathematically) Image: • Domain : compact; 2D, 3D; or 2D+t, 3D+t ⇒ video often: • Range : grey values, color values, “hyperspectral” values often: • Practical computing: domain and range are discretized • Domain “sampled” (pixels, voxels) • Range “quantized”, e.g., • Function piecewise constant or smooth interpolant 3 9/7/17 Image Examples (I) Grey value images Cone beam image in dentistry Windowing 0 256 0 4095 window entire grey value range medium-width window small-width window depicted ⇒ poor image contrast ⇒ good overall contrast, ⇒ high contrast for covering soft-tissue bone and teeth and bone Image Examples (II) 2D, 3D, … X-ray projection Electron Tomography (resolution: 1,5 nm) 4 9/7/17 Image Examples (III) static dynamic MRI of a head (resolution: < 1mm) Real-time MRI of a human heart (resolution 2mm / 50 ms) Image Examples (IV) Color images RGB image layers („color channels“) color image Color space: 3 dimensional Pixel values = coordinates in color space Light microscopy (in anatomic pathology) 5 9/7/17 Image Examples (IV) Vector images Flow mapping in cardiology Images (IV) Tensor images Diffusion tensors (2D slice in 3D) visualized by ellipsoids Fiber tracks 6 9/7/17 Visual Computing in Medicine 3DExcite - Living Heart – Powerwall ( Dassault Systemes ) Visual Computing in Medicine Acquisition, processing, analysis and rendering of all visual information (images/videos, 3D models) that arises during diagnosis, treatment and prevention. Requires techniques from • Image/video processing, pattern recognition, computer vision, machine learning • Computer graphics, visualization, computer vision, virtual and augmented reality, human computer interaction 7 9/7/17 Data in Medicine Medicine: science & practice of the diagnosis, treatment and prevention of disease, whether physical or mental. A medical treatment involves several processes, where the patient's health status is always the center of attention. Health Status • Medical history ⇒ Lots of Data! • Directly accessible parameters • Data from imaging E.g. in cardiology nowadays per standard examination: about 2 GB • Data from laboratory Medicine: Overall Process, Data Processing (Computer Aided) (Computer Aided) (Computer Aided) (Computer Aided) Diagnosis Therapy Planning Treatment Healing Prediction Initial State Changing State Changing State Predicted State Anamnesis Measurement Monitoring Monitoring Simulation Measurement Simulation Simulation Visualization Visualizaion Simulation Visualization Visualization Visualization 8 9/7/17 Simulation Personalized Simulations in Medicine In diagnosis: reveal information that cannot be measured, e.g., compute the loading of a knee joint for different types of movement, given the individual anatomy and the body weight. In therapy: plan and optimize treatments / surgeries e.g., enable surgeon to try different surgery techniques pre-operatively, given the current anatomical and physiological state In prevention: make long-term predictions of health state, e.g., depending on different life styles, given the current health state 9 9/7/17 Personalized Simulations in Medicine In chemical space mainly ordinary differential equations (ODEs) (systems biology) and stochastic differential equations (SDEs) In space & time mainly partial differential equations (physical) ⇒ Finite Element methods ⇒ Anatomical models required 10 9/7/17 Anatomy Reconstruction - an exemplary task of visual computing A Vision 11 9/7/17 Let these guys inspire our imagination… *) Dr. „Bones“ McCoy Spock Physician on the star fleet spaceship Enterprise *) Richard A. Robb, Biomedical Imaging, Visualization, and Analysis, Wiley-Liss, 2000 Dr. McCoy‘s Ultimate Healing Device A very compact handheld device: 1. Point it to the body of the patient; then the complete anatomic, physiological, biochemical and metabolic status is instantaneously determined and displayed. ⇒ “tricorder” 2. Place it on the diseased or injured region; then complete cure is effected. 12 9/7/17 Our Less Ambitious Aim We are satisfied with creation of a reconstructed 3D patient model suitable for planning, optimization and control of a therapy è Patient Models 13 9/7/17 Patient Models population average patient-specific patient-specific postmortem postmortem antemortem (!) real model virtual model virtual model anatomystuff.co.uk Teran, Sifakis, Blemker, … & Fedkiw Zachow, Muigg, Hildebrandt, Doleisch & Hege Creating and simulating skeletal muscle Visual exploration of nasal airflow. from the visible human data set. IEEE Trans Visual Comput Graph, 2009. IEEE Trans Visual Comput Graph, 2005. Patient-Specific Models Model anatomical è geometrical (this talk) functional è physical / mathematical / numerical (biomechanical, physiological, …) 14 9/7/17 Example Applications: Surgical Reconstruction How must bones be shaped? Sobotta © Lamecker, Zachow, Hege, Zöckler, Haberl: Zachow, Lamecker, Elsholtz & Stiller: Surgical Treatment of Craniosynostosis based on a Statistical Reconstruction of mandibular dysplasia using a statistical 3D Krebs 3D-Shape Model, CARS 2006 shape model. International Congress Series (Vol. 1281, pp. 1238-1243). Elsevier, 2006. Example Applications: Implant Design / Fitting / Individual Manufacturing fractures joint replacement Mccullochlaw.net dentures 15 9/7/17 Example Applications: Functional Simulation Tetrahedral heart model Beating heart Zhang Y, Bajaj. C. Finite element meshing for cardiac analysis. ICES Technical Report 04-26, University of Texas, Austin, 2004. Anatomical & Functional Models Functional Models systems of ordinary differential equations systems of partial differential equations ⇒ Requirements for Anatomical Models good geometric approximation of shapes good numerical approximation of functions 16 9/7/17 Anatomy Reconstruction: On All Length Scales Whole body MRI Blood vessels in brain, MRI angiography (7T) Trabecular structure of the radius bone micro-CT Pyramidal neuron from the hippocampus, CFM Golgi apparatus in cell, ET Reconstruction Pipeline 17 9/7/17 Images ⇒ Models Images • Anatomical information ⇒ anatomical models • Functional information ⇒ functional models Anatomy reconstruction • Identification and segmentation of anatomical units • Creation of (discrete) geometrical shape representations Anatomy Reconstruction Pipeline Image Data Image Filtering Image Registration Image Segmentation Surface Reconstruction Surface Improvement Volumetric Grid Generation 18 9/7/17 Denoising Improve results of later processing steps (edge detection) • Median filter • Anisotropic diffusion • Nonlocal means • Statistical methods • Machine learning Image Denoising with ML Use sparse coding combined with deep networks J. Xie, L. Xu, and E. Chen Image Denoising and Inpainting with pre-trained with a denoising auto-encoder Deep Neural Networks. In Advances in Neural Information Processing Systems, 341–349 (2012). BLS-GSM = Bayes Least Squares with a Gaussian Scale-Mixture KSVD = Dictionary Learning via SVD SSDA = Stacked Sparse Denoising Auto-encoder Feed-forward convolutional neural network K. Zhang, W. Zuo, Y. Chen, D. Meng, to separate the noise from the noisy image and L. Zhang. Beyond a Gaussian Denoiser: Residual Learning of Deep CNN for Image Denoising. IEEE Trans Imag Proc, 26:7 (2017), 3142 - 3155 Not yet applied to medical image data! 19 9/7/17 Image Registration Necessary when • Multimodal imaging is used • Acquisitions made at different times • Imaged organs are moving Overlay of 2 images in checkerboard pattern unregistered registered Image Registration Ingredients: • Set of allowed spatial transformations: rigid, affine, free, … • Similarity measure, based on corresponding features • Optimization procedure 20 9/7/17 Image Registration Literature: J Modersitzki: Numerical Methods for Image Registration, Oxford University Press, 2004 S Klein, M Staring, K Murphy, MA Viergever, JPW Pluim, Elastix: a toolbox for intensity- S Henn, K Witsch: Iterative Multigrid based medical image registrations, IEEE Trans Regularization Techniques For Image Matching, Med Imag 29:1, (2010) 196-205 SIAM J. Sci. Comput., 23:4, (2001), 1077-1093 Software: • Elastix toolbox, http://elastix.isi.uu.nl • FAIR, www.mic.uni-luebeck.de/people/jan-modersitzki • ITK - Segmentation & Registration Toolkit, https://itk.org Image Registration using ML S. Wang, M. Kim, G. Wu, D. Shen • Unsupervised Deep
Details
-
File Typepdf
-
Upload Time-
-
Content LanguagesEnglish
-
Upload UserAnonymous/Not logged-in
-
File Pages39 Page
-
File Size-