Fast Prototyping of Image Analysis Applications with Mathematica

RAPID PROTOTYPING OF BIOMEDICAL IMAGE ANALYSIS APPLICATIONS WITH MATHEMATICA

B.M. ter Haar Romeny, M.A. van Almsick

Eindhoven University of Technology, Eindhoven, the Netherlands Department of Biomedical Engineering

[email protected] [email protected]

Abstract  Complete (w.r.t. functionality)  Professional text, graphics, animations; We report on 2.5 years successful use of high level  Mathematical symbolic notation; rapid prototyping in education and research of  Platform independent biomedical image analysis, using Mathematica.  Easy extensions of operations to N dimensions

Introduction Typically the design is in two stages: after the development of the (mathematical) routine into a Digital acquisition techniques have caused an working prototype, a more clinical or industrial version explosion in the production of medical images, is / can be generated in C++, Java, etc. We use the especially with the advent of multi-slice CT and Visual ToolKit (VTK) and OpenGL for the interactive volume MRI. One third of the financial investments in 3D volume visualizations, as this is not contained in a modern hospital’s equipment are dedicated to Mathematica. imaging. Emerging screening programs add to this flood of data. The capabilities of many recent Implementation in the department computer-aided diagnosis (CAD) programs are compelling, and have recently lead to many new CAD We decided to promote the use of MMA for every companies [1]. This calls for many new algorithms for student project initiated, to establish rapidly a critical image analysis, and dedicated scientists for the job. mass of people with expertise. The second author, Image analysis software libraries abound [2], but authorized MMA trainer with over 10 years of unfortunately are often limited in functionality, are too experience regularly conducts interactive training specific, or need a rather dedicated environment and sessions for new batches of students. All student have a long learning curve. Today’s computer vision reports are generated as MMA ‘notebooks’, effectively algorithms are based on solid mathematics, requiring a combining a professionally looking text document with highly versatile, high level mathematical prototyping the annotated code. The University has one of the environment. We have chosen for Mathematica largest MMA licenses in the Netherlands, so the (MMA) by Wolfram Research [3], and describe the program could be installed on all systems, including successful results of the first 2.5 years of its use in the student and staff laptops for home use. Installation of training of biomedical engineers in image analysis. all campus software is eased through the public folders in MS Outlook. Special MMA pages on the group’s Software Requirements website [4] summarize all instructions, tutorials, software packages and latest news for staff and In 2001 the BME department established a new MSc students. track on ‘Medical Imaging’, including our group on Biomedical Image Analysis. As we started from For all second-year students (10 groups of each 8 scratch, no software infrastructure was present, and we students) a so-called ‘casus’ was developed: “Image made a choice for a new rapid prototyping analysis for pathology”. Duration 6 weeks, 50%. environment based on the following requirements: Students are challenged to find clever algorithms for the analysis of shape, number, texture and size of  A steep learning curve histological cell images. The many ‘inventions’  Functional programming possible, and the ease of programming their ideas in  Pattern matching MMA was very rewarding for most of the students. It  Typically short code turned out that especially the adoption of a functional  Integration of code and text programming style took longer, as this is a new and  Symbolic functionality unconventional way of programming for 2nd year  Interpreter (interactive debugging) students (e.g. no more loops, use of pattern matching).  Fast numerical functionality

Proc. of the Mediterranean Conf. on Medical and Biological Engineering MEDICON 2004, 31 Jul – 5 Aug 2004, Ischia-Naples, Italy.  The Visible Mouse: segmentation and visuali- 5 6 zation of high-resolution 3D 6.3 Tesla MRI data

10 9 (brain, heart) of the mouse for molecular imaging; 8  Segmenting the intervertebral space in high

1 resolution 3D CT data for individualized design of 7 vertebral disk implants;  Thickness color map on surface renderings 4 2 3  3D phantoms and realistic test images;  Computer-Aided Diagnosis (CAD) of lung 11 pathology by multi-scale texture analysis; Figure 1: Example of output of a cell classification In the 4th year a facultative course [6] is given by the project by 2nd year BME students: red outline for first author: “Front-end vision and multi-scale image acanthocyte cells, yellow for unknown or overlapping, analysis”. This course takes a multi-scale differential green for round normal blood cells . geometry approach to computer vision, based on bio-

mimicking mathematical models of the first ‘ Design oriented projects’ (DOP’s) are 3rd and 4th physiological stages of human visual perception. This year’s student projects where they solve a practical intensive 2-week course is also a national PhD course. problem in small groups. Duration: 6 resp. 12 weeks, The textbook of this course is completely written in 50%. Students successfully worked on the following Mathematica [3], and has an included CD-ROM with topics, all implemented and documented in MMA the full code / text. All topics can be interactively notebooks (which are all available at [4]): experimented with. During the interactive computer  Finding the hardly visible Adamkiewicz vessel in laboratories after the lectures students individually or 3D MRA datasets; in pairs study the tasks in the chapters of the book by  Image matching by mutual information Mathematica implementation. From the review forms maximization; collected it was clear that this lab was highly  Edge preserving smoothing for 2D and 3D data by appreciated. The learning curve was steep. PDE evolution (fig. 1.);

 Trabecular bone morphology statistics; Original Corrected 100 100  Volume visualization, perspective rendering; Field Subtracted 80 80

60 60 250 0 -250 40 40 -500 200

100 20 20 100 200 0 0 0 20 40 60 80 100 0 20 40 60 80 100

Original Corrected 500 500

400 400

300 300

200 200

100 100

20 40 60 80 100 20 40 60 80 100

Figure 3. 2D MRI background inhomogeneity removal by finding a second order polynomial subtraction surface by entropy minimization and gradient descent. Figure 2. Perspective 3D volume rendering of the ‘lobster’ dataset. In the 5th year Master’s projects of 9 months duration are carried out, most of them oriented towards  Volume visualization of color microscopy 3D algorithm development: images;  Catheter and guidewire detection in low-dose  Maximum Intensity Projection (MIP) visualization fluoroscopy images; for 3D color microscopy images;  Multi-scale optic flow analysis of the moving heart  Deblurring CT Gaussian slice thickness blur for wall from MRI tagging, using Lie derivatives; enhanced multi-planar reformat-ting;  Multi-scale level sets and watershed segmentation  Face recognition by Eigenfaces; for 3D datasets of the Visible Mouse; Active shape models for 3D shape variability of   Image retrieval by multi-scale toppoints; the normal and infarcted mouse heart;  Motion analysis by multi-scale toppoints;  Histogram equalization;  Perceptual grouping of edge contours for generic  MRI background equalization by entropy segmentation of biomedical data by multi-scale, minimization (fig. 3.); biologically inspired oriented filterbank analysis;

Proc. of the Mediterranean Conf. on Medical and Biological Engineering MEDICON 2004, 31 Jul – 5 Aug 2004, Ischia-Naples, Italy. MathVisionTools kernel size is strongly reduced. Gaussian blurring is L v v We are developing a rapid prototyping environment in described by the diffusion equation = 蜒. L , Mathematica for biomedical image analysis s v 禳抖 applications: MathVisionTools. The output of the where is the nabla operator 睚 , , . Diffusion student’s projects is moderated by the authors of this 铪抖x y z paper, and added to this collection of routines. Students is made inhomogeneous by the introduction of a always are guaranteed to use the latest release by conductivity constant, which decreases for larger edge reading the package code from the internet with strength: v v 蜒L. L MMA’s GetURL command. We consider to be on-line - L v2 v as the default situation while carrying out study or = 蜒.ek L . research. This option also turned out to be a convenient s option by sharing images or code in a collaborative  Calculus`VectorAnalysis`; project. SetCoordinatesCartesianx, y, z; L  Grad L x, y, z ; As image analysis often involves geometric reasoning,  pm im_,  _, k_  the use of differential geometric methods is abundant.  L.L  One of the key elements of the package is a multi- SimplifyDivE k2 L. scale, N-dimensional, speed optimized Gaussian Derivativenx_, ny_, nz_Lx, y, z differential operator convolution function, to take gDn im, nx, ny, nz,   2 2 2 multi-scale derivatives of discrete data to any order.  gDn im,0,0,1, gDn im,0,1,0, gDn im,1,0,0, 1  2    k 2  We also added the functionality to read DICOM files k  (the international / industrial image standard format for 2 2 k  2 gDn im, 0, 0, 1,  gDn im, 0, 0, 2,   medical images), including older .ani and .spi dialects, 2   2 k gDn im, 0, 2, 0,   2 gDn im, 0, 1, 0,  and returning all descriptive groups.    gDnim, 0, 2, 0,  4 gDnim, 0, 0, 1,  gDnim, 0, 1, 0, gDnim, 0, 1, 1,  A WebMathematica [9] server is installed, enabling the gDnim, 1, 0, 0, gDnim, 1, 0, 1,  design of a user-interface in a web browser, 4 gDn im, 0, 1, 0,  gDn im, 1, 0, 0,   2   communicating the MMA commands to a dedicated gDnim, 1, 1, 0,  k gDnim, 2, 0, 0,  server. MMA is an interpreter, so code cannot be 2 gDn im, 1, 0, 0,  2 gDn im, 2, 0, 0,   compiled and given away e.g. to doctors, but code can    be ‘published’ on the web. This could be an effective  peronamalikc1 im_,  s_,  _, k_, niter_ : mechanism to offer remote image analysis services,   Module , evolved  im; such as web-mediated computer-aided diagnosis for  Do evolved   s pm evolved, , k , niter ; hospitals.    evolved; Additional infrastructure Figure 4. Mathematica example to generate the PDE and code for 3D edge-preserving smoothing. The Image processing and computer vision algorithms command /. (Replace) replaces all occurrences of a typically require fast processors and large amounts of symbolic derivative into the numerical 3D Gaussian memory. At the TU/e all students are equipped with a derivative convolution function gDn. partly sponsored laptop when they enter the University

(currently 6800 students). Students which acquired Original scale  9 their laptop some years ago are faced with poor specifications for today’s jobs. Mathematica consists of two programs, a front-end and a kernel. The kernel can run on a remote computer. We implemented several rack mounted server PC’s with dual processor 2.8 GHz processors, running Linux, for remote Mathematica kernels. This solved the capacity problem with a minimal investment and great student’s satisfaction. Figure 5. Edge preserving smoothing (with the code of fig. 4) reduces speckle noise in an ultrasound image, A code example: while maintaining the edges for quantitative analysis or segmentation. Mathematica code can often be written in close similarity to the actual mathematical equations, even Discussion with the same mathematical symbolic notation. Edge preserving denoising (fig. 5) can be accomplished The implementation of a high level prototyping by an adaptive blurring strategy: in homogeneous programming environment has been an important step regions a large kernel is applied, and at edges the

Proc. of the Mediterranean Conf. on Medical and Biological Engineering MEDICON 2004, 31 Jul – 5 Aug 2004, Ischia-Naples, Italy. to facilitate especially the design phase of image analysis routines, both for education and research. Mathematica has turned out to be an excellent choice, References being a coherent program with both powerful symbolic and numerical capabilities. The symbolics enables the [1] CAD companies: www.r2tech.com, student to ‘play with math’ again, and the numerics www.deustech.com, www.cadxmed.com etc. now outperform many competitive matrix oriented [2] Computer vision software list: programs. www-2.cs.cmu.edu/afs/cs/project/cil/ftp/html/v- The code written in our department will be freely source.html. available for those labs that join us (on an exchange [3] Mathematica: www.wolfram.com. basis) in creating a high-level MathVisionTools box. [4] Biomedical Image Analysis Group: www.bmi2.bmt.tue.nl/image-analysis/ [5] TER HAAR ROMENY, B.M. (2003): ‘Front-end vision and multi-scale image analysis’, (Kluwer Academic Publishers, Dordrecht). [6] Front-End Vision course: www.bmi2.bmt.tue.nl/image- analysis/education/courses/FEV/course/index.html. [7] TER HAAR ROMENY, B.M. (2002): ‘Computer Vision & Mathematica 4’, Computing and Visualization in Science, 5, 1, pp. 53-65. [8] www.mathvisiontools.com. [9] www.wolfram.com/products/webmathematica [10] SUINESIAPUTRA, A. , FLORACK, L.M.J., WESTENBERG, J.J.M., TER HAAR ROMENY, B.M., REIBER, J.H.C., LELIEVELDT, B.P.F. (2003): ‘Optic Flow Computation from Cardiac MR Tagging Using a Multiscale Differential Method: A Comparative Study with Velocity-Encoded MRI’. Lecture Notes in Computer Science, vol. 2878, Randy E. Ellis and Terry M. Peters (Eds.), pp 483- 490.

Figure 4. Tissue classification of atherosclerotic plaque in the human arteria carotis from 5 different high resolution MRI acquisitions. The slices from the 3D datasets (top row) are registered, corrected for background inhomogeneity, and denoised by edge preserving smoothing (bottom row). Classification (left in rows) into 5 tissue types is accomplished by K- means analysis of a 5-dimensional feature space (top).

Acknowledgement We thank the BMIA students for their enthousiasm in pioneering and useful feedback.

Proc. of the Mediterranean Conf. on Medical and Biological Engineering MEDICON 2004, 31 Jul – 5 Aug 2004, Ischia-Naples, Italy.