An Open Source Framework for Efficient Development Of

GIMIAS: An open source framework for eﬃcient development of research tools and clinical prototypes

Ignacio Larrabide a,b,∗, Pedro Omedas b,a, Yves Martelli b,a, Xavier Planes b,a, Maarten Nieber b,a, Juan A. Moya b,a, Constantine Butakoﬀ b,a, Rafael Sebasti´an b,a, Oscar Camara b,a, Mathieu De Craene b,a, Bart H. Bijnens c,b,a, and Alejandro F. Frangi b,a,c

Center for Computational Imaging and Simulation Technologies in Biomedicine (CISTIB) aNetworking Biomedical Research Center on Bioengineering, Biomaterials and Nanomedicine (CIBER-BBN), Barcelona, Spain bUniversitat Pompeu Fabra, Barcelona, Spain cInstituci´oCatalana de Recerca i Estudis Avan¸cats(ICREA), Barcelona, Spain ignacio.larrabide,pedro.omedas,[email protected]

Abstract. GIMIAS is a workflow-oriented environment for addressing advanced biomedical image computing and build personalized computational models, which is extensible through the development of application- specific plug-ins. In addition, GIMIAS provides an open source framework for efficient development of research and clinical software prototypes integrating contributions from the Virtual Physiological Human community while allowing business-friendly technology transfer and commercial product development. This framework has been fully developed in ANSI-C++ on top of well known open source libraries like VTK, ITK and wxWidgets among others. Based on GIMIAS, in this paper is presented a workflow for medical image analysis and simulation of the heart.

Key words: Biomedical imaging, cardiovascular modeling, Virtual Phys- iological Human, personalized simulations, open source software

1 Introduction

The integrative multi-scale and multi-disciplinar community that tries to under- stand the human patho-physiology is internationally known as “IUPS Physiome Project” [1,2] or, in the European context as “Virtual Physiological Human” (VPH) [3]. Besides the strategic character that it has regarding the priorities of the 7th Framework Programme of the EU, the VPH is of great relevance in the current context given its integrative vision of the physiological and computational knowledge. In this perspective, a key challenge in the development of integrative and predictive models for the human physiology is the creation of 2 GIMIAS: Graphical Interface for Medical Image Analysis and Simulation computational tools that enable the collaborative work between scientists, industry and health care to bridge the gap between scientiﬁc discovery and clinical treatment. An essential condition for the success of the VPH is the development of software tools that ensure methodology integration. As examples we can site all multi-scale integration, access and exploitation of large repositories and databases, processing of data with state-of-the-art algorithms, creation of personalized models, data fusion and visualization. In this paper we present GIMIAS (Graphical Interface for Medical Image Analysis and Simulation), a workﬂow-oriented environment that has been specially tailored for fast development of biomedical imaging and modeling software prototypes. GIMIAS provides an open source framework for the development of research tools and clinical prototypes, allowing the integration of contributions from the Physiome community while allowing business-friendly technology transfer.

2 Related work

Current tools (see Table 1) have their own strengths and weaknesses, mainly linked to the core skills of the research group behind it. However, there is not currently, a single application that fulfills all the requirements of all possible VPH applications. For this reason, the VPH community is steadily progressing towards establishing data, modeling and communication standards that ensure interoperability between different tools rather than creating a single application that accounts for all possible needs. Furthermore, there is also a need for a Free Open Source Software platform that allows efficient development of prototypes, that requires minimal training time or prior knowledge to get started. What GIMIAS brings to the community is a common framework to help on data manipulation and visualization in which processing algorithms can be easily integrated. In this environment new algorithms and models (developed by the user or provided by an existing library) can be easily integrated in more complex processing workflows. Furthermore, GIMIAS and all the libraries on which it relies are Free Open Source Software. In this way, different teams using the platform, either from academia or industry, can interact and share modules easily. The main goal of this platform is to transfer novel technologies into clinical environment by the fast development of prototypes.

3 Speciﬁcations

Taking into account all the requirements for the platform, GIMIAS architecture has been designed to accomplish the following speciﬁcations. The following non- functional requirements, i.e., criteria usable to judge the system other than its functionalities, should be addressed by GIMIAS: – Modularity: its functionalities should be modular allowing ﬂexibility in the way that they are combined to facilitate exchangeability and reuse. Graphical Interface for Medical Image Analysis and Simulation 3

– Simplicity: to allow technologists and scientists to easily integrate their algorithms in the platform in with small amount of time and eﬀort. – Flexibility: to support the development of research and clinical prototypes with diﬀerent degrees of complexity. Also, binary (compiled) and/or inter- preted (script-based) plug-in development should be possible requiring an even smaller amount of prior knowledge to integrate new features. – Interoperability: to cope with the increasing amount of image processing and modeling tools available and to ensure maximum reuse, it is of crucial importance that this software is capable of inter-operating with others.

Name Distribution license Brief list of features BrainVISA Open source software. CeCILL Li- - scientific visualization cense. - signal and image processing CMGUI Open source software. Mozilla - scientific visualization Public License. - scripting capabilities - model personalization - signal and image processing MedINRIA Free for non-commercial use. - scientific visualization - model personalization - signal and image processing MeVisLab Basic version free for non- - scientific visualization commercial use. SDK version - scripting capabilities requires a commercial license. - plug-in architecture - image processing OpenMAF Open source software. BSD-style - generic application framework License. - scientific visualization - model personalization - image processing ParaView Open source software. BSD-style - scientific visualization License. - scripting capabilities - plug-in architecture Slicer 3D Open source software. BSD-style - scientific visualization License. - plug-in architecture - image processing Table 1. Summarized list of current software development projects focused on medical image processing and end-user software for model personalization. The brief feature list is based on the description provided by their distributors. The following list contains the corresponding websites for the tools described above where further details can be found: www.brainvisa.info, www.cmiss.org/cmgui, www- sop.inria.fr/asclepios/software/MedINRIA, www.mevislab.de/, www.openmaf.org/, www.paraview.org, www.slicer.org.

In order to illustrate how GIMIAS can be used for the implementation of a particular workflow, let us use as an example an application for cardiac modeling. On this regard, the treatment of biomedical data often involves (sometimes large) work-flows that require different data processing tools. This is particularly important in the case of cardiac modeling, where the use of computational tools 4 GIMIAS: Graphical Interface for Medical Image Analysis and Simulation

Fig. 1. Left: Schematic overview of the plug-in based architecture of GIMIAS. General infrastructure services are used by the different plug-ins according to the specific application. Plug-ins can be changed to create new workflows. Right: Classes to be developed, and their associated complexity and interaction, for the creation of a new plug-in. For the classes Widget and DataView the developer can choose either implementing them or using the standard implementation provided by the framework. for image and signal processing, surface and volume mesh editing, creation of numerical models and data visualization is required.

GIMIAS framework users GIMIAS, as a fast prototyping open source framework, has been developed think- ing in two different user profiles: 1. Researchers who want to test methodologies and easily create prototypes for validating the practical feasibility of research concepts. Using the proposed framework, they can create new plug-ins that incorporate their methods and algorithms, allowing potential end-users (e.g., medical doctors, clinical scientists, among others) to assess them. 2. Scientific developers (from academic or industrial environment) that need to create fast and cheap prototypes for research tools and/or clinical products. Moreover, by having a BSD licence, GIMIAS is appropriate for promoting interaction between industrial and academic software development teams. GIMIAS provides a simple Application Programming Interface (API) that permits, with minimal effort, to trial methodologies and algorithms in the form of end-user applications. Also, prototypes developed within/for GIMIAS can be tested by end users in real scenarios and with real data at early development stages, providing very useful feedback as well as reducing the time required to get new concepts from research to the clinical environment.

4 The GIMIAS architecture

GIMIAS has been designed to be extended via the addition of plug-ins. Plug- ins are extension units (e.g., dynamic link libraries or shared objects depending Graphical Interface for Medical Image Analysis and Simulation 5 on the platform) for an application whose architecture allows functionalities to be introduced by users or programmers at well-deﬁned places after it has been installed. In GIMIAS, plug-ins are software entities that can be created with minimal experience in programming for extending the application by providing speciﬁc features in a modular way.

Plug-in based software architecture

GIMIAS provides an API for the plug-ins to interact with the application core and with other plug-ins. The classes that can be extended for the creation of a new plug-in are: Widget, SceneView, DataEntityList, FrontEndPlugin and Processor (see Figure 1). Depending on the complexity of the desired plug-in, not all these classes need to be developed. The minimal set of classes needed to create a plug-in with new behavior are WorkingData (containing the data used by the plug-in), FrontEndPlugin (which describes the main functionality of the plug-in) and Processor (executes an algorithm given one or more input data and generates one or more output data). Very brieﬂy, the classes that need to be used and/or extended for the creation of plug-ins are (see Fig. 1):

– DataEntity: This class stores and encapsulates a single data object (e.g., medical image, transformation, surface mesh, model simulation result, etc.). – DataHolder: This class allows objects to register as observers of data, for instance, to be notiﬁed when the data is changed. – WorkingData: This class holds a reference to the DataHolders associated to the data required and is currently used by the plug-in (selected input, output, etc.). – DataEntityList: This class contains a list populated with the data entities currently available for processing in the application. The input data that will be processed by the plug-in and also their outputs are stored in this list. – FrontEndPlugin: This is the base class for all the plug-ins, every new plug- in must extend this class. This base class provides access to the rendering classes, but delegates the creation of the rendering scene to the extending class’ implementation. – Processor: This class executes one algorithm and given on or more input data and generates one or more output data (usually coming from the DataEntityList). A plug-in may contain more than one Processor class. No data is stored inside the Processor.

Regarding data processing, each plug-in uses its own (one or more) Processor classes that encapsulate the application logic indicating which algorithm needs to be executed on the data. External algorithms (user defined or furnished by a library) are called by a Processor after the required input data have been specified. After the execution of an algorithm, the Processor notifies (through the DataHolder) the corresponding observers that new data is available. These data can be rendered or processed depending on the application. It is the re- sponsibility of the plug-in to determine how the processing pipeline continues. 6 GIMIAS: Graphical Interface for Medical Image Analysis and Simulation

In this way, the plug-in developer is responsible for defining how the end-user operates over the data and visualizes it. DataEntity encloses and represents 4D working data in a single entity that can be modified, transformed and rendered by any plugin. This class allows plugins to share data, encapsulating its format conversion thus facilitating the exchange and flow data. For its simple architecture, the sole specialization/implementation of three classes is enough for creating new plug-ins. In this way, new features can be integrated very fast. The architecture is flexible to support plug-ins with different complexity, going from simple (e.g., data is processed with a specific algorithm and the results are visualized with standard viewers) to complex (e.g., the entire user interface can be refurbished to create a new data interaction and visualization application). In addition to its plug-in API, the framework provides basic services which can be used by the plug-ins:

– scientiﬁc visualization through standard screen layout (i.e., 3D plus orthog- onal slicing), – multimodal (MRI, SPECT, MSCT, US, PET) image processing, – heterogeneous data management (images, polylines, surface geometries, volumetric meshes), – standard I/O formats (others can be added by the users).

All these services are accessed trough the FrontEndPlugin class. Further details on other features and plug-ins provided by the framework can be found at www.gimias.org. Regarding interoperability with external tools, the framework allows to easily integrate diﬀerent format readers and writers. A closer integration (e.g., in runtime) with other tools is possible by developing a speciﬁc plug-in for this matter.

Third-party libraries

GIMIAS has been entirely developed in ANSI-C++ and is distributed under BSD-licence. The reason to make the software open source is to facilitate and foster the cooperation and interaction between diﬀerent research groups and with industry for technology transfer. This licence model together with its plug- in architecture make it ideal for this kind of interaction. A proper selection of third party libraries is crucial for the long term continu- ity of the software. For this reason only BSD-licensed libraries with consolidated and active user communities where selected. For the graphical user interface, wxWidgets 1 was considered. The mesh manipulation is based on The Visual- ization Toolkit (VTK) 2, an open source, freely available software system for 3D computer graphics, image processing, and visualization with a large and estab- lished user community around the world. For user interaction and visualization

1 www.wxwidgets.org 2 www.vtk.org Graphical Interface for Medical Image Analysis and Simulation 7 of data, the Medical Imaging Interaction Toolkit (MITK) [4] is used. For han- dling data in DICOM standard, DCMTK library is used 3. For image processing operations The Insight Toolkit (ITK) 4 was chosen.

5 Example workﬂow in GIMIAS

In this section we analyze in detail a case study based on image analysis and modeling of the heart. The objective of this case study is to show the capability of GIMIAS to integrate different algorithms in a single workflow. To this end, patient images in DICOM format are loaded. Next the images are segmented using an anatomical atlas of the heart. The last step involves generating a personalized computational model of the heart containing myocardial fiber orientation, the Purkinje system and different mesh labels (endo- and epicardium) for the subsequent electrophysiological simulation. This is addressed in a single user application that goes from the image to the personalized simulation model.

DICOM reading plug-in

The objective of the DICOM plug-in is to let the user browse medical images stored in DICOM format, allowing to load 2D, 3D and 4D data (Fig. 2). Once the data is loaded, it is organized using a tree viewer to facilitate visualization of patient, study, series and time point data. Using this plug-in, the user can inspect diﬀerent studies and navigate through the slices and corresponding time points. The DICOM plug-in creates a volumetric or 3D+t image from the selected DICOM data including the slices and time points selected by the user. After that, the image can be easily rendered or located in the DataEntityList where it becomes visible to the rest of the application for further processing.

Cardiac image analysis plug-in

In order to obtain a personalized geometry of the heart from a medical image, image segmentation based on an anatomical heart atlas is used [5]. This atlas is ﬁrst initialized using anatomical landmarks such as the left ventricle apex, the aorta and mitral valve ring (Fig. 2). Once the heart atlas has been positioned within the image, the plug-in allows the user to adjust the shape of the atlas using the image as reference and thus, semi-automatically segment the heart. Using this feature, the user can deform the heart model (by simply dragging the surface interactively by using the mouse) and adjust the shape to the image. Automatic segmentation using active shape and image appearance models is currently under development and will be integrated in the prototype in the future. In the case of 4D images, it is possible to segment the whole sequence of images starting from any given time frame. The plug-in allows the user to

3 dicom.oﬃs.de/dcmtk 4 www.itk.org 8 GIMIAS: Graphical Interface for Medical Image Analysis and Simulation obtain a number of quantitative measurements of the heart including the ejection fraction, wall motion, wall thickness, wall thickening, among others, during the manual segmentation process to give immediate feedback to the user. After the segmentation, the result can be analyzed using Bull’s eye plots (AHA 17-segment model) for the diﬀerent variables such as wall thickening, radial wall motion, among others. As will be shown below, the resulting geometrical models are used for the generation of personalized simulations of the cardiac electrophysiology.

Cardiac modeling plug-in Before being able to use the anatomical model to perform electrophysiological or electromechanical simulation it is necessary to generate a proper FEM model. It is necessary to mesh the surface obtained from the segmentation step, and produce a volumetric fine mesh which will be made up of tetra- hedral elements. Volumetric mesh quality is ensured by controlling both the maximum distance between neighboring nodes and the radius-to-edge ratio to assure regularly-shaped elements. In keeping with recommendations from lit- erature high-resolution meshes with inter-node distance ≤ 500 µm to avoid physiologically inaccurate numerical effects such as wavefront warping and other artifacts. Excitation and recovery sequences are influenced by the direction of myocardial fibers, which determine the longitudinal and transversal orientation of the anisotropic conductivity of intra- and extra-cellular media. As current imaging techniques are unable to non invasively extract myocardial fiber orientation from the beating heart, mathematical models are used to replicate the rotation of fibers across the ventricular wall. In GIMIAS, the fiber orientation is calcu- lated for every element of the mesh, using a mathematical formulation based on the work of Streeter [6]. In this way, the familiar helical structure of fibers was reproduced, with orientations rotating from 50◦ at the endocardium to −60◦ at the epicardium. The Purkinje System (PS) is also an important part of the system that helps the fast and synchronous electrical propagation of the impulse coming from the sinus node. Since the reconstruction of the PS from specific patients is not feasible, a fiber network has been defined according to the descriptions by Durrer. The PS is represented as a 1D cable model that is placed upon the endocardial surface, containing the bundle branches and 100 Purkinje cables that expanded from the bundle terminals towards the ventricular apex continuing onto the left and right free walls. The procedure to include the PS consists in a manual delineation of the fibers in the heart atlas, and subsequently a propagation of the network to each mesh. The propagation is done during the segmentation of the ventricles, where the atlas is deformed to match the specific geometry of the patient heart, fitting at the same time the PS embedded in the atlas.

6 Conclusions GIMIAS is an environment specially tailored for the creation of work-ﬂow applications in the ﬁeld of biomedical image computing and personalized simulation. Graphical Interface for Medical Image Analysis and Simulation 9

The architecture of the underlying framework is based on plug-ins through which its core functionalities can be extended. Other services such as PACS connection, parallel computation through grid middleware, and scripting are in the development roadmap of the framework. In this work we presented a cardiac image analysis and simulation application developed within this environment. In the presented workﬂow, GIMIAS provides the capability of segmenting images of the heart and allows improving the outcome manually when the segmentation is not perfect. GIMIAS provides an open source framework for eﬃcient development of research and clinical software prototypes within the Physiome community while allowing business-friendly technology transfer and commercial product development.

7 Acknowledgements A complete list of contributors to the analysis, design and implementation of this platform can be found at www.gimias.org. This research has been partly funded by CIBER-BBN (VPHTk project), the industrial and Technological Develop- ment Center (CDTI) under the CENIT programme (CDTEAM project) and the European Community’s Seventh Framework Programme (FP7/2007-2013) under the grants agreement n.224495 (euHeart Project) and n.223920 (VPH NoE). Dr. O. Camara and R. Sebastian acknowledge grant support from the Spanish Ministry of Research and Innovation, under a Ramon y Cajal and Juan de la Cierva Research Fellowship, respectively.

References

1. Hunter, P., Robbins, P., Noble, D.: The IUPS Human Physiome Project. 445(1) (2002) 1–9 2. Hunter, P., Borg, T.: Integration from Proteins to Organs: The Physiome Project. Nature 4(3) (2003) 237–243 3. Fenner, J., Brook, B., Clapworthy, G., Coveney, P., Feipel, V., Gregersen, H., Hose, D., Kohl, P., Lawford, P., McCormack, K., Pinney, D., Thomas, S., Jan, S.V.S., Waters, S., Viceconti, M.: The EuroPhysiome, STEP and a roadmap for the virtual physiological human. 366 (2008) 2979–2999 4. Wolf, I., Vetter, M., Wegner, I., Böttger, T., Nolden, M., Schöbinger, M., Hasten- teufel, M., Meinzer, H.P.: The Medical Imaging Tookit. 9 (2005) 594–604 5. Ordas, S., Oubel, E., Leta, R., Carrera, F., Frangi., A. In: A statistical shape model of the heart and its application to model-based segmentation. Volume 6511. In Proc. SPIE Medical Imaging, San Diego, CA, USA (2007) 6. Streeter, D.: Handbook of Physiology, Section 2: The Cardiovascular System. In: Gross morphology and fiber geometry of the heart. Volume Volume I: The Heart, Chapter 4. American Physiological Society (1979) 61–112 10 GIMIAS: Graphical Interface for Medical Image Analysis and Simulation

DICOM plugin

Cardiac segmentation and quantification plugin

Cardiac simulation plugin

Fig. 2. Cardiac modeling workﬂow as implemented in GIMIAS. From top to bottom are presented the DICOM, the Cardiac segmentation and quantiﬁcation, and the Cardiac simulation plug-ins. After desired images are selected from the time sequence, an atlas is used to segment the images and adjust the segmentation result where required. Then, from the resulting heart geometry, a model for simulation of the heart electrophysiology is created. Finally, the propagation front in a selected region of the cardiac muscle is presented.