Medipo – Medical Image Processing Online a Web Based Decision Support and Educational Tool for Medical Image Processing

MedIPO – Medical Image Processing Online A Web Based Decision Support and Educational Tool for Medical Image Processing

Bengisu Cagiltay, Bilkent University, Turkey

Mert Ege Can, Bilkent University, Turkey

Imge Gokalp, Bilkent University, Turkey

Caner Sezginer, Bilkent University, Turkey

Abstract: Today, Magnetic Resonance Imaging (MRI) technology makes very serious contributions to medical science in order to make various decisions before surgery. However, surgeons working on MRI are required to make serious decisions about the location of tumor and their volume, size and shape. They have to make plans to minimize the damage to the patient while separating tumorous areas from the healthy tissues. This planning can be done manually by working on a large number of MRI with different sections or by means of a software. However, gaining these skills is a challenge for the surgical education programs. In this study a system is developed to analyze the patient data, detect and record abnormal masses with a semi-automated image processing tool for pre-surgery decision support and educational purposes. Through this database, this study aims to provide case-based try-and-error type practice alternatives to the surgical residents. MedIPO provides a web based system which allows its users to upload, view and analyze medical images online by providing a semi- automated image processing system. By such a practice it is aimed to improve the beginner level of surgical residents’ skills on better detecting the locations of abnormal masses to improve efficiency for the pre-surgery decision making process. This study summarizes the main concept of the system development, the used algorithms as well as the user interface for analyzing the MRI data for educational purposes.

I. Introduction and Background of the Study Surgeons are required to analyze the MRI data of a patient and make a surgical plan prior to the surgeries such as brain surgery. However, there are not many educational tools to provide try-and error type of training for the surgery education programs. Today currently there exists some medical image processing systems available in the market such as Osirix Lite, 3D Slicer, AnalyzeDirect, Online DICOM Viewer and MicroDICOM. However, when compared to MedIPO, these systems have some limitations for educational purposes when their prices and functionalities are considered. For instance, Osirix Lite is a desktop application that allows to open and view DICOM [1] type images. The software has features such as image processing, image visualization, advanced post-processing techniques in 2D and 3D, fast performance, FDA & CE certification [2]. However, as it is a desktop application it has to be installed the computer for being used. Additionally, its cost for each installation might create a serious limitation for the educational purposes and it does not provide any guidance for the decisions being made. Another software 3D Slicer is a free desktop software package for image analysis and scientific visualization of DICOM images [3]. It has features like image processing, scientific visualization (3D) and image analysis. However, it does not automate image processing features, it has a lower quality user interface and since it is a desktop application this software also has limitations for educational purposes. Analyze Direct is a software that presents image processing, visualization and analysis tools [4]. However, its image processing features are manual and as it is a desktop application it has a cost for each installation, therefore it has limitations for educational purposes. Online DICOM Viewer is a web-based application that users can post medical DICOM images to view and save [5]. As it is free to use, web based and has visualization features, it is more beneficial for the educational purposes when compared to the previous products. However, it does not have any image processing features and it is not suitable for advanced processes which limits its educational use. MicroDicom is an application for primary processing and preservation of medical images in DICOM format [6]. MicroDicom includes viewing and saving medical images in DICOM format, brightness/contrast control, zooming and panning DICOM images, medical image processing operation. However, it has manual image processing features thus it is considered slow for responses when compared to automated systems. After conducting analysis of these tools, it has been decided that a system which does not require any installation of additional software and hardware, which is web-based and free of charge, which allows to store, compare, download and also provide automated image processing capabilities would allow much more accessibility to medical doctors and speed up their decision support and learning processes. Accordingly, Medical Image Processing Online (MedIPO) is developed to satisfy these requirements stated, in order to better support the medical education systems.

MedIPO System MedIPO is a web based system which allows its users to upload and analyze MR images quickly and easily, online. The target audiences of this system are medical doctors. The system aims to provide an online viewer and image analyzer for medical doctors by providing a detailed and intractable interface. MedIPO can be used for pre-surgery planning, practice and educational purposes. The users are required to upload files in DICOM format [1]. For ethical reasons, the uploaded MR images will be automatically anonymized by the system. The system enables a medical doctor to specify the region of interest and it conducts a semi-automated segmentation of this region (e.g. ventricles of the brain, cortices, cerebellum, tumor, etc.). The system also allows the user to enter manual annotation and corrections in addition to the segmentation results. The final packages of the software are based on the Client-Server relations. The client and server packages share the Model-View-Controller (MVC) design pattern and the sub layers of the packages are designed according to the MVC structure. A visual representation of the package structure is presented in Figure 1.

Figure 1: Package Structure Since MedIPO system is a web based application, the system does not require any direct external hardware components. The hardware-software interaction occurs between the web browser and the machine. The application requires internet connection on the machine in order to provide data transaction between the server and the client. As seen in Figure 2, the web browser interacts with Java servlets over HTTP requests.

Figure 2: Hardware Software Mapping The main aim of MedIPO software is to provide a system with zero-footprint, which indicates that the system does not require any client side install or download. This allows the users to view and edit the medical images online, purely over the browser. In order to achieve zero-footprint software, all the data related to the system such as login information and DICOM images to be stored in the server’s internal disks.

II. Algorithms and Design Decisions Two of the major algorithms used for the software are named as Magic Wand and Magic Grid. A brief description for both of the algorithms will be explained in this section.

Magic Wand Magic Wand algorithm is a simple algorithm used for border detection of a selected region of interest. This algorithm works by taking an input from the user which specifies the region of interest. The pixels around the region of interest are collected in this algorithm and according to a specified threshold, the neighboring pixels are either selected or not. The same action is repeated for every new pixel added to the region of interest. One of the major design decisions based on the Magic Wand algorithm is due to the success rate of this algorithm. Magic Wand algorithm has a high success rate in clear, closed boundary images. However, since tumors might not always visualize clearly, in some images this algorithm did not provide highly accurate results. Therefore, one major design decision during the algorithm development of Magic Wand was to develop a separate algorithm that is less dependent on the color differences of pixels, which is the following algorithm Magic Grid.

Magic Grid Magic Grid algorithm is based on the image processing algorithm of Superpixels (Slic). Magic Grid uses the Slic algorithm as a baseline, the original image is divided into Slics. Later, these Slic’s are presented to the user interface in order to interact with the user. The user selects the region of interest by clicking on the Slic’s and creates a region called “mask”. This mask is used in order to reference the neighbor images (i.e. previous and next images) or the original image itself. By using this mask, the region of interest is transferred to the previous and next images by adjusting the pixel values. The algorithm checks whether a pixel of a superpixel overlaps more than 50%, and the pixel’s color difference is less than the specified threshold. If both of these conditions are true, the new superpixel is selected in the neighboring images. As a design decision made during the algorithm development of Magic Grid, the primary algorithm consisted a different design from explained above. The primary design used the center points of Slic’s with combination of the threshold value. However, in order to provide more accurate results while using the Magic Grid algorithm, the design is changed to the approach explained above.

III. System Workflow

Server Workflow Server-side workflow is responsible for analyzing the process of a generated response after a request is received from the client. According to the request type, the data layer will decide on the relevant flow for the request which might be one of the following: Magic Wand, Magic Grid or Upload. In all three paths, after the request-specific processes are completed, a response is generated and returned to the server. At this point, the server makes decisions based on the response and the relevant information is shared with the client.

Figure 3: Server Workflow Client Workflow The client-side workflow is responsible for analyzing the path that a request follows. The request sent from the server to the client is managed in three ways depending on the type of the request. The first path leads to the service manager actions which are register, login and logout. The second path leads to the user interface actions, which is selecting the region of interest on the image. The third path leads to the transfer manager actions which are file management, upload and download. After the relevant path to follow on the workflow is identified the corresponding data analysis are conducted in each path. According to the results of the data analysis, a request is sent to the server and finally a response from the client is presented.

Figure 4. Client Workflow

Activity Workflow In order to demonstrate the simplicity of the application, the final activity workflow is presented in the image below (Figure 5). As demonstrated in the activity workflow, the system has minimum amount of interaction required by the user, thus the primary purpose of simplicity is achieved in the system workflow. The main activity workflow starts with the home page, followed by uploading images and viewing image history straightforward. Later the user is presented with two options for image segmentation: Magic Wand and Magic Grid. According to the segmentation decision, the activity workflow continues with editing images and according to satisfaction of the user, the activity workflow concludes with downloading the images.

Figure 5: Activity Workflow

IV. System Interface and User Manual The system interface is design in an easy to use manner showing the navigation bar/menu is under the header that includes helpful links to the most used pages as seen in Figure 6. On the left side of the navigation bar, pressing the “Upload” link will take the user to the upload page and if the user is logged in, the “Image History” link show the last uploaded images of the user.

Figure 6. Welcome Page Upload page (Figure 10) allows the user to upload multiple image files to the system for MedIPO to work on. User can choose the files they want to upload by clicking “Choose Files” button and browsing the files from their computer.

Figure 7. Upload Page After choosing the files they want, they can submit the files by clicking “Submit Files” button which will send the user to the Image History page. Image History page (Figure 8) allows the user to see the last images they uploaded. If the user has logged in, the system will remember the last images they uploaded even if the user did not upload anything this time.

Figure 8. Image History Page The user can see every image individually by clicking the arrow buttons on the sides of the image slider. MedIPO has 2 different algorithms that are accessible to the user: magic wand tool and a magic grid tool. Both can be accessed by clicking on “Magic Wand” and “Magic Grid” buttons. User is also able to delete their uploaded image history by clicking the “Delete History” button.

Figure 9. Magic Wand Page Magic Wand page (Figure 9) is a tool that basically lets the user select different parts of the uploaded image by the color. The image can be selected by clicking on the image on the left and an area will be selected in red. The selected area (Figure 10) can be selected again by clicking on the image at any time.

Figure 10. The selected area (in red) The selected area can be increased and decreased by clicking on the “>+” and “-<” buttons. User can also zoom in and out the image by using “Zoom IN” and “Zoom OUT” buttons. The selection can be erased anytime by clicking the “CLEAR” button. When the user is happy with the selected area, “APPLY Selection button can be pressed to apply the selection to the neighboring slices accordingly. All the images and selections are available to download by clicking the “Download ZIP” button. Magic Grid page (Figure 11) is a tool that basically lets the user select different parts of the uploaded image by dividing it into multiple grids.

Figure 11: Magic Grid Page The image can be selected by clicking on any of the grids on the image on the left and a grid will be selected in red. If the user wants to select multiple grids, they can enable that feature by clicking on the “Current Selection: Single Region”, which will turn into “Current Selection: Multiple Region”. The selection type can be changed anytime by pressing the button again. The selected grid (Figure 12) can be selected again by clicking on the image at any time.

Figure 12. The button region of Magic Grid Tool The area of the grids can be increased and decreased by clicking on the “<” and “>” buttons. Similarly, the magic wand tool threshold can be increased and decreased by clicking on the “<” and “>” buttons. The three toggle buttons “GRID: ON” “BORDER: ON” and “COLOR: ON” changes the visibility of the edited regions’ grid, border and color. “PAINT LEFT” “PAINT ALL” and “PAINT RIGHT” button allows the user to reflect the edited region changes to the other slices. The selection can be erased anytime by clicking the “CLEAR CURRENT” button for clearing the selections of current slice of “CLEAR ALL” button for all of the slices. The toggle button “Current Selection: Single Region / Multiple Region” allows the user to edit the super pixels one- by-one or in a multiple way. When the user is happy with the selected area, “SAVE Selection button can be pressed to save the selection to the neighboring slices accordingly. All the images and selections are available to download by clicking the “Download ZIP” button (Figure 13).

Figure 13. The selected grid in Multiple Selection setting (in red)

V. The Educational Use of MedIPO The developed system has features to guide and better train surgical residents for abnormal mass detection and pre-operation decision making processes. Accordingly, this system is an example of computer-based support for the decision making processes of the surgeons to improve their decision making skills during the surgical education programs. Through such tools, technology enhanced training environments can be provided for the surgical residents. Once the MRI data sets with detected tumor information is enriched in the system, its possible benefits for the trainees would be increased. As each case require different decisions, through such systems wide range of cases can be provided to the surgical residents to better provide experience on different possible cases. As the anatomical structures of each patient may have differences as well as the location of the tumors can be very different requiring different strategies for the operation, such case based experiences are very critical in the surgical education programs. In this study, a tool capable of providing image analysis to a wide range of medical cases is developed in order to enhance the learning experiences of medical students and provide decision support for medical doctors. We believe that, the classical surgical education programs can be enriched with such technologies by creating a rich case-based hands on training opportunities. The proposed systems also can be enriched by feedback systems to better guide the decisions of the trainees by evaluating the taken decision and giving feedback about each decision. Such case-based technology enhanced training environments potentially improves the surgical performance by providing a decision support system prior to the real operations and also should increase performance of the educational programs by shortening the training durations and improving the quality. The system is currently designed according to the requests of medical doctors and during the development phase the team received feedback from experts in the field. In the future we are planning to evaluate this model by a team of medical doctors and trainees. The evaluation results will help us to improve system and adapt it to the current training programs of the neurosurgery departments of medical schools. We believe that by improving and integrating such systems in the curriculum of medical schools, the current training programs could be enhanced which in turn will provide several benefits to the society as well as the education providers.

References [1]"DICOM Standard", Dicomstandard.org, 2017. [Online]. Available: http://www.dicomstandard.org/. [Accessed: 5- Nov- 2017]. [2]"OsiriX DICOM Viewer | The world famous medical imaging viewer", Osirix-viewer.com, 2018. [Online]. Available: http://www.osirix-viewer.com. [Accessed: 02- May- 2018]. [3]"3D Slicer", Slicer.org, 2018. [Online]. Available: https://www.slicer.org/. [Accessed: 02- May- 2018]. [4]"AnalyzeDirect : 3D Medical Image Analysis Software for Research", AnalyzeDirect, 2018. [Online]. Available: https://analyzedirect.com/. [Accessed: 02- May- 2018]. [5]"Online DICOM Viewer for Free", Onlinedicomviewer.com, 2018. [Online]. Available: http://onlinedicomviewer.com/. [Accessed: 02- May- 2018]. [6]"MicroDicom - Free DICOM viewer and software", Microdicom.com, 2018. [Online]. Available: http://www.microdicom.com/. [Accessed: 02- May- 2018].