An Architecture for Implementing Customizable Medical Image Processing Systems

Athanasios M. Demiris, Carlos E. Cardenas S., Manuela H. Makabe, Hans-Peter Meinzer

Deutsches Krebsforschungszentrum, Heidelberg, Germany

Abstract nario in system development. According to this scenario smaller systems addressing specific topics are created sepa Monolithic image processing systems containing a superset of rately on demand and may be integrated into a larger system in imaging algorithms are difficult to use and require specialized a' component-like manner. Creating specialized components knowledge ofimage processing. Thus they increase the work addressing specific problems instead of all-purpose systems load ofmedical personnel instead ofmaking the work situation enforces user centered design and development as well as easier. Customizable medical image processing systems on the encapsulation of overcomplicated functionality, which is feasi other hand may be easily adapted to address various problems ble, only when a concrete and restricted application domain is . in the medical image processing domain integrating only the addressed. necessary subset ofimage processingfunctionality presented in an intuitive way. One possible instantiation ofthat scenario is when an end user using an extensible platform system, which we characterize as a In this work we present an architecture for creating customiza customizable medical image processing system, contacts soft ble image processing systems for the medical domain. We ware developers every time the functionality needs to be address three major topics: 1.) easy, goal-oriented customiza extended to target a new problem, like volumetric measure tion ofimaging systems by using a generalized algorithm model ments for pathologic changes in the liver, cancer detection in and repository, 2.) dynamic, data-oriented parameterization of mammography, visualization ofthe beating heart etc. In order the selected algorithms and 3.) semi-automated generation of for developers to react to these needs in a shorter time and thus user interface components for each new algorithm to be reduce the period oftime from demand to delivery (lead time), inserted in an imaging system based on cognitive ergonomics. an advanced, easy to customize building tool is needed. A fur We conclude with the presentation ofan initial implementation ther advantage ofthe use of such a development tool is the con ofthe architecture in form ofan object-orientedframework for sistency of the developed applications. the creation ofcomponents for customizable medical imaging When customizing or creating a new software system for the systems. medical imaging domain there are some aspects that delay the response to a request for such a system. An example could be Keywords the request for a volumetric system in liver surgery. Such a sys Object-Oriented Frameworks; Algorithm Repository; Imaging tem requires methods for retrieving and displaying the image Systems; User Interfaces; Ergonomics data, algorithms for the segmentation, i.e. identification of healthy and pathologic tissue in the images, the selection of the Introduction appropriate algorithms for image processing, subsequently the estimation of optimal parameter sets for these algorithms to The image related part ofthe work of physicians and especially operate in a meaningful way on the given data, and fmally the radiologists and surgeons has been continuously enhanced by creation of a user interface for the resulting system. An archi computer supported components within the past decades. One tecture that supports developers in this process offers means for goal ofthe developers ofsuch systems was to ensure flexibility reducing the time spent with all these steps. In this paper we are by supplying their end users with a maximum of functionality presenting such an architecture and its partly implementation in in order to cover for all possible cases that might arise in the form of an object-oriented framework [1]. future. The results were monolithic systems containing too many features and being difficult to use. The level on which the Materials and Methods use of these systems was taking place was closer to the image processing part and farther from the actual end user needs. The work presented here started with the creation ofa volumet The advances.in software engineering within the past years and ric system for liver resection planning. We developed VolMes, especially in object-oriented technology gave rise to a new see- a system for image segmentation and quantitative measure-

1100 1101 A.M Demiris

Developer of image processing routines Medical End User

query algorithms or query/store parametersets insert new algorithm

stausuca/observauons

GUI creation support Repository and dynamic parameter calculations

store/retrieve

L-f-~--F-.f-----+ Database management system

Abstract database connection interface

Figure 2 - The global scenario ofthe architecture ments in contrast enhanced images of liver tissue (CT or MR) Instead ofcustomizing VolMes statically by extending and rec containing tumors or metastatic regions (see figure 1). ompiling we decided to design a dynamic solution that would The system contains image and DICOM-information display, address the quick replacement of algorithms by looking up semiautomatic and manual segmentation facilities. It was cre alternative algorithms' according to the new goal and a flexible ated according to ergonomic design criteria and in continuous estimation of default parameters for these algorithms every time cooperation with clinicians (user centered design). in order to be the kind ofdata set changes. These aspects are dealt with in our smoothly integrated in clinical routine. . architecture by an algorithm repository and a component for dynamically estimating parameters for various types of image data of different anatomical regions. Subsequently we decided to integrate the new algorithms with pre-customized user inter face components for the control of these algorithms. We address this problem in our architecture with a model for auto matically creating user interface elements according to ergo nomic criteria for every new algorithm. In figure 2 we show the all-round scenario and in the next paragraphs we explain how the corresponding parts ofthe architecture are designed. A generic medical image processing algorithm model and repository When trying to select the algorithms needed to solve a given problem the developers usually look up possible solutions in their own "arsenal" (or the ones oftheir teams). Usually devel opers get acquainted with a small set of algorithms they use over and over and thus may skip other solutions. The documen tation of the algorithms takes place in a rather informal way thus allowing discrepancies in the way it is done among various developers. Less experienced developers sometimes do not Figure 1 - A screenshotofthe image segmentation andvolumet- have the chance to profit from existing solutions due to the ric system VolMes complete lack of documentation especially when the developers The system was subsequently used in other medical domains of existing algorithms have left the team. (e.g. heart segmentation) which led to the conclusion that for A central algorithm repository is definitely a way to overcome these cases other default settings for the parameters of the tools these barriers and reduce the amount of time needed for the and in many cases completely different tools were needed. selection of algorithmic material. Such a repository allows a 1102 Imaging andImage Management faster lookup of algorithms according to several criteria espe repository along with the algorithm. When a new algorithm is cially goal-oriented and not only implementation-oriented ones. used or an existing one is to be applied to a new case there are For an efficient repository to be created a flexible data model is no default values for the algorithm's parameter set (its signa neededIn order to ensure a general character of such a reposi ture). For these cases we developed a component estimating tory model we concentrated on extending existing models of parameter sets in an iterative process. This component looks up general algorithms foundin programming languages and speci- initial settings in the repository or calculates random ones. It fications of standards like the reflective! object-oriented pro applies the algorithm to the image data and evaluates the result gramming languages Smalltalk [2] and Java [3] and the ing binary image according to a series ofmeasurements. specification of the Object Database Standard by OMG 14] as We introduced a domain model containing relative measure well as the CORBA [5] specification. ments and anatomical landmarks for the elimination of the In our model we introduced the notion ofthe algorithm as a spe interpersonal variation for various regions ofthe body and con cialization ofa software building block. We described the algo tinuouslyextend the model to include new regions. These meas rithm in terms ofits signature (the parameters and return value) urements include: and its context. By context we mean both the kind of and the size ofthe segmented object relative to an easy to detect restrictions on the data to be processed (such as the format or organ (e.g. liver to thorax in CT), the datatype of a digital image) as well as the execution envi center ofinertia ofthe segmentation result, ronment (such as the machine architecture, a shell or a net • extend ofthe segmented object, browser in case ofan applet needed for execution, etc.), In fig ure 3 we are showing a very rough sketch ofthe model using orientation, the Unified Modeling Language notation by Booch, Rumbaugh connectedness (number of holes in the segmentation and Jacobson described in [61to give an overview of the most result) and important dependencies between the biggest classes or groups • homogeneity (based on gray-values). ofclasses in the model. The resulting values for a segmentation are evaluated using a When inserting a new algorithm in the repository the developer fuzzy reasoning engine. The reason for the introduction ofthis is requested to enter information documenting the algorithm in engine was the necessity to formalize semi-empirical rules for. a well defined manner. Thus the lookup of this algorithmic the combination of the above mentioned measurements and material may be done in a flexible way according to different conclude whether and which ofthe estimated values need to be criteria. When similarities to existing algorithms are detected recalculated. while entering a new algorithm the developer is asked by the We are evaluating the success ofthese measurements for differ system to explain the relationships ofthe new algorithm to the ent regions ofthe body. Additionally we are considering a sta existing ones if there are any. It is also possible to define pre tistical evaluation component collecting values for successful and post-execution conditions in order to be able to use the new segmentations carried through by users in order to ensure that algorithm with others minimizing side effects when using them there will be initial values for more application cases even in in a chain. cases where the developers of algorithm do not supply default values for parameters during algorithm insertion in the reposi tory. Automated generation ofuser interface elements for medi cal image processing algorithms When retrieving an algorithm from the repository developers usually want to test the results ofalgorithm application on their data. Therefore a user interface for the variation ofthe parame ter values is needed. This user interface maybe reused when the algorithm is selected for integration into a larger system, espe cially when the system offers dynamic extension mechanisms (as it is the case with VolM~s,or t~e "extensible t~le~radiolog~ Figure 3 - A rough sketch ofthe object model ofthe image algo- system and image viewing station" CHILf 'l7]).··· IIi 'order .. to' rithm repository reduce the time spent with developing ad hoc one way user interfaces many times for the testing and integration of algo Determination ofoptimal parameter sets rithms we introduced a component that automatically creates When an existing algorithm is to be used in a known case some user interfaces for algorithms inserted in the repository. default values for the parameters can be retrieved from the . This component takes as input the signature of the algorithm and proposes a series ofpossible interaction models (e.g. is the 1. Reflectivelanguagesseparatebetweena base levelwherethe applicationobjectsare describedandcoded and a metalevelwhere value for a parameter obtained by a mouse click or a dedicated descriptions of the base levelobjectsand their interactionsare con entry field in the interface etc.), This generation module can be tained.Reflectiveprogrammingsystemsexploitmajorpartsof the used as is or customized to support different look-and-feels systemitself such as the parts describinghow algorithmsare repre sentedand implementedin the correspondinglanguage. accordingto the system where the algorithm is to be integrated. 1103 A.M Demiris The generated interface employs the algorithm either in form of hide all aspects not necessary for solving the actual problem by, an executable or as a call to a dynamically linked object. offering default solutions or high level interfaces to low level For the creation of the user interface we use the Model-View functionality. Thus the developers of systems do not have to Controller interaction schema (known as MVC) [8]. According deal with every detail of the resulting system and concentrate to this schema the application for which the user interface is to only on the necessary aspects. be created makes up the model layer and is fully independent of We selected object-oriented frameworks as the most appropri any presentation aspect. The static part ofa graphical user inter ate way to implement the above described architecture since face to the model is the view. One or more views can be defmed they are easier to extend and customize than a development for a model, e.g. the interface to an algorithm may be an image environment. that the algorithm modifies along with some interaction ele ments (e.g. buttons) or it could be a statistical representation Results like a histogram or both. Finally the interaction with the user is captured on the controller layer. The controller layer defmes the We started implementing the described architecture as an response ofthe system to mouse-clicks, text entries etc. object-oriented framework. All components have been tackled The automated generation is based on the model given by the in different development environments varying from Smalltalk, algorithm and creates the corresponding view and controller. to C++ and Java. Each sub-component of the framework has The view is determined in a lookup manner, e.g. when a been applied and tested in a stand-alone manner in our depart numeric parameter is to be displayed a predefmed element is ment. There is no statistical evaluation ofthe gains in develop created while different elements are created for textual or binary ment effort and time after the introduction of each component parameters. For the controller there is more than one possible up to now but we have planned an extensive evaluation after the alternative which are presented to the developer inserting the integration of all parts into a single framework. The fmal inte- algorithm for selection. In the default case for each image gration development is taking place in Java. . parameter an image displaying element along with some basic The algorithm repository is completed and we are now dealing controls (like level/window, zoom and next/previous slice for with the mapping to an abstract database interface using JDBC, volume data) is created while all other parameters are set by which will allow us to integrate a number of existing database dedicated elements. In other interaction models parameters can management systems for the storage ofthe algorithms. be selected to be obtained by mouse clicks (e.g. coordinates ofa seed point). The parameter estimation component is currently being tested in different cases. We have focused our efforts on the liver anat In order to ensure ergonomic design as described in [9] we omy both in CT and MR since the measurements developed for introduced an additional layer in the MVC schema that checks that anatomic region have been extensively tested in clinical for violation of basic ergonomic rules when creating the inter applications. The next area already being addressed is the one face to an algorithm. This layer contains basic rules concerning ofthe human heart. coloring, positioning etc. This layer "filters" the way views and controllers are created and thus we named the resulting model The automated creation of user interface components is com MVCC (Model-Yiew-Qontroller-Qonstraints) as described in pleted and now the refinement phase is introduced. Due to the [10]. use of platform independent user interface libraries we accom plished a full platform independence creating interfaces for all A possible implementation with object-oriented frameworks OSFlMotifand Windows 95/NT systems. The connection to the The architecture described here results in a system that supports MVCC will take place after the porting of the current imple developers of dedicated solutions that can be integrated into mentation ofthe latter from Smalltalk to Java. larger end-user systems. There are many ways to implement this architecture. One possibility would be to create a dedicated Discussion development environment. This solution though is difficult to implement and maintain. It also requires taking into account a We introduced a new architecture for the fast development of number of possible other development tools that the developer solutions to dedicated imaging problems in medicine that may may want to use and introducing interfaces to each one ofthem. be integrated into larger customizable systems. We tested the A second option is to implement the architecture in form ofan feasibility ofall parts ofthe system in a stand-alone manner and advanced software library that the developer may integrate into are currently integrating and evaluating the impact ofthis archi any other development environment and customize on demand. tecture as a complete object-oriented framework. Object-oriented frameworks are the advancement of simple At the time being our targeted customizable imaging system is software libraries and dedicated event-loop systems, containing VolMes. VolMes is restricted on medical image analysis and all generic aspects ofa solution in a given domain [11]. In con does not take into consideration communication or printing trast to program libraries they also include dynamic aspects ofa aspects and marginally addresses data security. Our next goal is solution and in contrast to event-loop systems they contain fully to create new plug-in components for the CHILI system which customizable interaction models. Frameworks can be custom successfully deals with all these topics. ized and/or extended to address specific needs. They usually 1104 Imaging andImage Management Conclusion ing Massachusetts, 1997 [7] Egelmann U, Schroter A, BaurU, Schroeder A,WemerO, We proposed an architecture for the development of adaptive Wolsiffer K, Baur HI, Goransson B, Boralv E, Meinzer image processing systems in medicine. The scenario presented HP Teleradiology System MEDICUS. in: Lemke HU, here may reduce the time from request to delivery ofa new ded Vannier MW, Inamura K, Farman AG (Eds.) CAR "96: icated imaging system (lead time). Computer AssistedRadiology. Elsevier, Amsterdam 1996 Acknowledgments [8] Rumbaugh J. Modeling Models and Viewing Views: A The authors wish to thank Marc Roger Gopfert, Dr.'Geraid Look at the Model-View-Controller Framework. Journal Glombitza and Antje Schroeder for their continuous contribu 'ofObject-Oriented Programming, 7 (2), 14-20, 1994 tions, support and enhancement suggestions. [9] Boralv E, Goransson B, Olsson E, Sandblad B. Usability and Efficiency: the HELlOS Approach to Development of References User Interfaces. Computer Methods andPrograms in Bio medicine 45 (Suppl.), 47-64, 1994 [1] Lewis T (ed.) Object-oriented application frameworks. [10] Demiris AM, Meinzer HP. Cognition Based Development Manning Publications, Greenwich 1995 and Evaluation ofErgonomic User Interfaces for Medical Image Processing and Archiving Systems. Medical Infor [2] Goldberg A, Robson D. Smalltalk-80 The Interactive Pro matics, in print gramming Environment. Addison-Wesley, Reading MA 1995 [11] Taligent Inc. Leveraging Object-Oriented Frameworks Taligent Inc., Cupertino CA, 1993 . [3] Grand M, Knudsen J. Java Fundamental Classes Refer ence O'Reilly & Assoc., Sebastopol CA 1997 Address for correspondence [4] Cattell, RGG (ed.) The Object Database Standard: Athanasios M. Demiris ODMG-93 Release 1.1 Morgan Kaufinann Publishers, Deutsches Krebsforschungszentrum 1994. Div. 0805 Medical and Biological Informatics [5] Object Management Group, Inc. Common Object Request 1mNeuenheimer Feld 280 Broker Architecture andSpecification, COREA Revision 2 D-69120 Heidelberg, GERMANY Framingham, Massachusetts 1995. e-Mail: [email protected] URL: http://mbi.dkfz-heidelberg.de/ [6] FowlerM, Scott K. UML Distilled - Applying the Stan dard ObjectModeling Language Addison-Wesley, Read-