FRIEDRICH-ALEXANDER-UNIVERSITÄT ERLANGEN-NÜRNBERG TECHNISCHE FAKULTÄT • DEPARTMENT INFORMATIK
Lehrstuhl für Informatik 10 (Systemsimulation)
Investigation and Implementation of Visualization and Reconstruction Options for Breast-CT
Simon Schmidt
Bachelorthesis Investigation and Implementation of Visualization and Reconstruction Options for Breast-CT
Simon Schmidt Bachelorthesis
Aufgabensteller: Prof. Dr.-Ing. Harald Köstler Betreuer: Dr. David Heinemann (AB-CT) Bearbeitungszeitraum: 1.6.2020 – 2.11.2020 Erklärung:
Ich versichere, dass ich die Arbeit ohne fremde Hilfe und ohne Benutzung anderer als der angegebenen Quellen angefertigt habe und dass die Arbeit in gleicher oder ähn- licher Form noch keiner anderen Prüfungsbehörde vorgelegen hat und von dieser als Teil einer Prüfungsleistung angenommen wurde. Alle Ausführungen, die wörtlich oder sinngemäß übernommen wurden, sind als solche gekennzeichnet.
Der Universität Erlangen-Nürnberg, vertreten durch den Lehrstuhl für Systemsimulati- on (Informatik 10), wird für Zwecke der Forschung und Lehre ein einfaches, kostenloses, zeitlich und örtlich unbeschränktes Nutzungsrecht an den Arbeitsergebnissen der Ba- chelorthesis einschließlich etwaiger Schutzrechte und Urheberrechte eingeräumt.
Erlangen, den 1. November 2020 ...... Abstract
The nu:view Breast-CT is the worlds first spiral Breast-CT. While already in clinical use, the system is still under on-going development. In the current clinical workflow, the acquired scans have to be reconstructed and then transferred to the clinical network before they can be reviewed on a workstation in another room. As the necessary amount of data for a single scan is huge, this can take long times. The goal of this thesis was therefore, to improve on those weak points by investigating viewing options which could be used in close proximity to the scanner. As the range of viewing solutions can reach from small local software to full-sized cloud solutions, an overview about different viewing concepts is given. Following this, the requirements and use cases for a close proximity viewing solution are defined exactly based on customer feedback. A promising way to reduce the amounts of processed data, is to select volumes of interest. Instead of the full scan, only those specific re- gions have to be reconstructed in higher resolution. To achieve those goals, different existing viewing solutions are compared, based on the defined requirements. Then, a proof of concept in software and hardware is imple- mented. This proof of concept is integrated into the default nu:view pipeline consisting of the Breast-CT Scanner and mulitiple workstations. The experimental implementa- tion is evaluated based on different criterias. It is shown, that noticeable improvements in reconstruction time and reconstruction size can be achieved. Contents
List of Figures iii
List of Tables iv
List of Listingsv
List of Abbreviations vi
1 Introduction1 1.1 Background ...... 1 1.2 Goals ...... 1 1.3 Scope of Thesis ...... 1
2 Clinical Workflow3 2.1 DICOM Standard ...... 3 2.2 PACS Systems ...... 3 2.3 nu:view Breast-CT ...... 3
3 Viewing Concepts5 3.1 Overview ...... 5 3.2 Operational Purposes ...... 5 3.2.1 Diagnosis and Post-Processing ...... 6 3.2.2 Early Quality Checks ...... 6 3.2.3 Reducing Amounts of Data ...... 7 3.3 Viewing Types ...... 7 3.3.1 Overview ...... 7 3.3.2 Pre-PACS Viewing ...... 8 3.3.3 Post-PACS Viewing ...... 9 3.3.4 Cloud Viewing ...... 10 3.4 Viewing Pipeline ...... 11 3.4.1 Overview ...... 11 3.4.2 Phase 1: Input ...... 11 3.4.3 Phase 2: Display ...... 12 3.4.4 Phase 3: Editing & Post-Processing ...... 14 3.4.5 Phase 4: Output ...... 14
4 Requirements 16 4.1 Retrospective ...... 16 4.2 Desired Purposes ...... 16 4.2.1 Clinical Evaluation ...... 17 4.2.2 Other Feedback ...... 17 4.2.3 Conclusion ...... 17 4.3 Non-Functional Requirements ...... 18 4.3.1 Overview ...... 18 4.4 Functional Requirements ...... 18 4.4.1 Overview ...... 18 4.4.2 Input: Local DICOM Studies ...... 19 4.4.3 Input: Command Line Interface ...... 19 4.4.4 Display: MPR ...... 19 Contents ii
4.4.5 Editing: VOI Selection ...... 20 4.4.6 Output: Export VOI Coordinates ...... 20 4.5 Comparison of Tools ...... 20 4.5.1 Overview ...... 20 4.5.2 RadiAnt DICOM Viewer ...... 21 4.5.3 3DSlicer ...... 21 4.5.4 ImpactView ...... 21 4.5.5 Conclusion ...... 21
5 Proof of Concept 23 5.1 Implementation ...... 23 5.1.1 Overview ...... 23 5.1.2 Pipeline ...... 23 5.1.3 Scan Monitoring ...... 25 5.1.4 ImpactReconCLI ...... 26 5.1.5 Coordinate Extraction ...... 28 5.2 Integration ...... 31 5.2.1 Hardware ...... 32 5.2.2 Network Topology ...... 32 5.2.3 AWS Configuration ...... 33 5.2.4 VWS Configuration ...... 33 5.2.5 RWS Configuration ...... 34
6 Evaluation 35 6.1 General ...... 35 6.2 Software Quality Measurements ...... 35 6.2.1 Functionality ...... 35 6.2.2 Reliability ...... 36 6.2.3 Usability ...... 36 6.2.4 Efficiency ...... 37 6.2.5 Maintainability ...... 38 6.2.6 Portability ...... 38 6.3 Conclusion ...... 39
References 41 List of Figures iii
List of Figures
1 nu:view Scanner ...... 4 2 nu:view Workflow ...... 5 3 Pre-PACS Viewing ...... 8 4 Post-PACS Viewing ...... 9 5 Cloud-PACS Viewing ...... 10 6 Viewing Pipeline ...... 12 7 Multiplanar Reformation ...... 13 8 Basic Pipeline ...... 23 9 ImpactView Selection UI ...... 24 10 Basic VOI Reconstruction GUI ...... 28 11 RIA Orientation ...... 29 12 Network Layout ...... 33 13 Example of VOI Reconstructions in Different Resolutions...... 36 14 Reconstruction times for different VOIs ...... 37 15 Reconstruction sizes for different VOIs ...... 38 List of Tables iv
List of Tables
1 Requirements for Different Purposes ...... 19 2 Software Comparison ...... 20 3 AWS Hardware ...... 32 4 RWS Hardware ...... 32 List of Listings v
List of Listings
1 Creation of FileSystemWatcher ...... 25 2 Example Realization of Monitoring ...... 25 3 Connecting an Action to an Event ...... 26 4 Reconstruction Example ...... 27 5 Shortcuts Example ...... 27 6 Batch Parsing Example ...... 27 7 Necessary Additions to LayerManager.h ...... 30 8 Creating and Connecting an Export Button ...... 30 9 Extraction of VOI Values ...... 30 10 Calculation of Voxelcounts for each Resolution ...... 31 11 Save VOI Values as Textfile ...... 31 12 Setting Up a scheduled Task ...... 34 List of Abbreviations vi
List of Abbreviations
AWS ...... Acquisition Workstation CLI ...... Command Line Interface DICOM ...... Digital Imaging and Communications in Medicine MTRA ...... Medical-Technical Readiology Assistant PACS ...... Picture Archiving and Communcations System ROI ...... Region of Interest RWS ...... Reconstruction Workstation VOI ...... Volume of Interest VWS ...... Viewing Workstation 1 Introduction 1
1 Introduction
1.1 Background
In todays world digital imaging systems are a constantly developing standard in every modern hospital. More and more analog medical devices are replaced by their digital substitutes. Additionally, virtually all patient data is processed digitally nowadays. From billing processes to the scheduling of appointments, most analogue processses have been replaced by their digital counterparts. Similar to other business areas or companies one can hardly imagine a modern hospital without any digitalized workflows. While this brought a lot of advantages in managing and storing the data, it also created a need for reliable and secure server-, storage- and communication architectures. Especially when considering digital visualization options for medical images, for exam- ple CT reconstructions, it is necessary to process huge amounts of data. As a result of this development, the de facto standards PACS and DICOM emerged. Those two systems are providing the foundation for the data management of most modern hospitals. The PACS use case lies in managing, delivering and archiving image data - which is stored, retrieved and transferred according to the DICOM standard and file format.
1.2 Goals
For medical device vendors this technical evolution leads to a difficult situation when trying to provide the best usability and the most optimal workflow to the customers. To create a product which fulfills all needs regarding technical functionality, regulatory requirements, usability and integration into the clinical workflow, the vendor has to consider lots of different software solutions, hardware setups and process pipelines. Based on those preconditions, the goal of this thesis is to investigate options for the implementation of a visualization feature of the nu:view Breast-CT system in clinical use. This includes the raw visualization function as well as the possibilites to reduce the amounts of data which has to be processed.
1.3 Scope of Thesis
The reconstructed scans are reviewed on a diagnosis workstation which is usually lo- cated in a different room than the scanner. The scope of this thesis is to investigate options for viewing the reconstructed CT images in immediate proximity to the scan- ner. As the market for third-party viewing solutions is pretty big, the work will focus on comparing existing visualization solutions based on the mentioned goals, technical re- 1 Introduction 2 quirements and implementation options. By prioritizing and evaluating those criterias, the most appropriate software will be selected. Building on this selection, a proof of concept will be implemented and integrated in an experimental setup. An evaluation of this prototype will happen afterwards. 2 Clinical Workflow 3
2 Clinical Workflow
2.1 DICOM Standard
For transferring and archiving digital medical images, the "DICOM" (Digital Imaging and Communications in Medicine) Standard was developed. It was first publicated in 1993 and defines a file format as well as network protocols to save and transmit digital images and additional information 1. It is widely adopted in todays clinical world and enables the interoperability between different medical devices from different manufacturers. Most picture archiving and communcation systems (PACS) are based on DICOM and would not be possible without, as described in [10, Chapter 1]. For the scope of this thesis, DICOM will be used exclusively as a file format for saving reconstructed images of the raw scans.
2.2 PACS Systems
The term "Picture Archiving and Communications Systems" is used to describe a sys- tem consisting of multiple components that handle the digital imaging workflow. Those components include the imaging devices itself, a network for transmitting the image data, workstations for viewing the images as well as archives for storing and retrieving them [6]. The default way of handling those interactions is the DICOM standard. While the nu:view Breast-CT becomes a part of the PACS, as soon as it is integrated into the clinical environment, the term PACS will be used as a collective term for the clinical network and its devices outside of nuView.
2.3 nu:view Breast-CT
The nu:view Breast-CT (Figure 1) is the world’s first spiral breast CT. It was developed by AB-CT Advanced Breast-CT GmbH which started as a spin-off from the institute for medical physics under the guidance of Prof. Willi A. Kalender. It delivers 3D images with high isotropical resolutions and offers an improved patient comfort compared to common mammography. Nu:view consists of multiple components:
• nu:view Breast-CT Scanner
• Acquisition Workstation (AWS)
• Reconstruction Workstation (RWS)
1For additional information on the scope of DICOM use [7, Chapter 1.1] 2 Clinical Workflow 4
Figure 1: nu:view Scanner
The CT device itself is controlled by the AWS. The AWS is a workstation which runs the "ScanCommander" software. ScanCommander allows the radiologists or medical- technical radiology assistants (MTRA) to initiate scans as well as configuring specififc parameters like reconstruction profiles, scan length and others. Especially important in the context of this thesis are the reconstruction profiles which represent "quality presets". Easily spoken, they are representing different resolutions in which the recon- struction of the raw data can be done. A higher resolution reconstruction comes with the downside of a much longer reconstruction time, as well as a much higher file size and therefore a much higher transfer time and network load. Currently there are two profiles available for clinical use: "Standard" and "HighRes". During this thesis how- ever, a third, low resolution preset "Preview" will be used in addition to them. This preset is only available for experimental purposes. After a scan is done, the resulting raw data is transferred from the AWS to the RWS. The RWS runs the "ReconServer" software. The ReconServer handles the reconstruc- tion of all incoming raw data to create the high quality 3D images for diagnostic purposes in DICOM file format. After the reconstruction of a scan is completed suc- cessfully, it sends the image data to the clinical network (PACS) for archiving or re- trieving it. In its current state is necessary to retrieve the image data from the PACS at a diag- nosis workstation for viewing it. This workstation has to be equipped with a DICOM conforming viewing solution. This diagnosis workstation is not included in the nu:view system. This pipeline is visualized in Figure 2. 3 Viewing Concepts 5
Figure 2: nu:view Workflow
3 Viewing Concepts
3.1 Overview
The market for DICOM viewing solutions is pretty big and chaotic. They can range from small, locally installed software to full-sized cloud solutions as well as from full- featured editing software to very limited viewing-only solutions. The purpose of this chapter is to give a general overview about the different use cases, to describe the different types of viewing that exist and to give an overview about the most common features of viewing software. As mentioned earlier in the text, during the clinical workflow a CT scan is being trans- ferred between a variety of devices. In the context of a device specific viewing solution, this opens multiple options where the visualization can take place. This position in the clinical pipeline can be relevant for multiple reasons, especially when considering different operational purposes. Not every operational purpose can be fulfilled at every position, so defining the most relevant purposes is crucial. Based on those purposes the best fitting option (referred to as "Viewing Type") can be chosen. In this section, an overview about a few potential purposes and the most common viewing types will be given. This will lead the foundation for selecting and prioritizing them in section 4 which opens up the possibility to particulary define a specific set of required features.
3.2 Operational Purposes
While in some hospitals, a central viewing solution is already integrated, this isn’t the case for all hospitals. This requires device vendors to come up with their own solutions or integrations to fulfill the needs of their customers and deliver an easy to use all-in- one package. On the other hand, if a solution is already implemented, it is required to make sure the device is compatible to the used solution. The spectrum of those solutions can be broad, ranging from single workstation implementations on a local basis to full-sized, device independent cloud environments. While this compatibility is often implicitly guaranteed if the device is working according to the DICOM standard, other disadvan- tages may still lead to chosing another solution over the already existent solution of the 3 Viewing Concepts 6 hospital. Especially if the desired purpose requires viewing capabilities before sending the data to the clinical network, a device specific solution can be a valid option. For this reason the operational purposes have to be defined clearly so that a well justified decision can be made. During the last years clinical studies and installation tasks, Advanced Breast-CT man- aged to get a lot of oral feedback from clinical staff members. Of special importance are features, requested by people who are working with nu:view on a daily basis, more specifically radiologists and radiologic technologists. Based on those experiences some regulary mentioned purposes for a viewing solution will be introduced below.
3.2.1 Diagnosis and Post-Processing
Right now, the usual reconstruction procedure is creating a set of multiple, recon- structed two-dimenional slices. Those slices are reconstructed using filtered back pro- jection. While this is a good way for creating general purpose reconstructions, some use cases may benefit from additional post-processing options to further enhance the visual clar- ity of specific areas or tissues. An often used post-processing algorithm that comes to mind is the maximum intensity projection (or its counterparts minimum- or average intensity projection). This would allow for emphasizing different types of tissue for easier diagnosis, based on the goal of the specific examiniation. A viewing solution which is able to deliver a big variety of post-processing options could be advantageous for the diagnosis workflow. Thus, this would not come without problems. A viewing solution focusing diagnostic purposes has to fulfil a lot of regulatory requirements (e.g. CE/FDA approval and more). In addition, this use case is in most cases already covered by the clinical diag- nosis workstation. It has to be well considered if a viewing solution of this scale and effort is really necessary in close proximity to the scanner.
3.2.2 Early Quality Checks
A problem in the current workflow, which is often mentioned by users of nuView is the possibility of a preview reconstruction. Right now, the options for reconstruct- ing the scan, as well as viewing it are very limited. Only two reconstruction profiles are released for clinical use. Both of them, Standard and HighRes are aimed towards delivering images for clinical diagnosis. While this may be sufficient for the diagno- sis workstation, this implies that the quality of the scan is secured in advance. Even a standard resolution reconstruction takes multiple minutes from scanning to recon- structing to transferring the scan to the diagnosis workstation. Without any possibility to preview the reconstructed images, a scan which does not fulfil all requirements will lead to a huge amount of lost time. 3 Viewing Concepts 7
One solution for this problem could be the use of a "Preview" scan profile. This profile could represent a quick low quality reconstruction, in a resolution which fulfills the bare minimum for doing some basic quality checks. Example criterias which could be evaluated in this step could be the correct positioning of the patient, the non-existence of motion artifacts and data consistency. All of them can be sufficently examined even with low resolution reconstructions. In an optimal setup, viewing this preview recon- struction would happen directly at the AWS, the RWS or, if this is not feasible, at a workstation in the same room as those systems. This would allow for an immediate quality check and acquisition of a new scan, depending on its results.
3.2.3 Reducing Amounts of Data
Building further on that, another potential use case comes to mind. Combining the possibilities of a fast low resolution reconstruction, as well as the minimization of the physical distance between the scanning workstations and the viewing solutions would allow to reduce the network and storage load on the workstations and the PACS. If a rendered view of the current scan is available next to the scanner, this may allow for discarding bad quality scans right before sending them to the PACS. As a result, network- as well as storage load could be reduced in amounts proportional to the amount of the bad quality scans. Additionally, the executing radiologist would not have to wait for the PACS to finish the scan transfer before being able to view it at the diagnosis workstation. Another way of reducing the amounts of data is to reduce the scan size itself. This could be done by selecting a volume of interest (VOI) in the viewing solution, which could then be reconstructed in a higher resolution. As a result, less time is required for the scan reconstruction as only a specific area has to be reconstructed in high res- olution, instead of whole scan. As the possibility of pursuing this goal is highly dependent on the actual implementa- tion of the viewing solution, especially the position in the pipeline where the viewing takes place, this purpose has to be especially well considered before selecting the desired viewing type.
3.3 Viewing Types
3.3.1 Overview
Throughout the course of the last chapter, the position of the viewing in the clini- cal workflow stood out as a very prominent and fundamental precondition. It has an particulary high impact on the operational purposes which may (or may not) be po- tentially realizable. Based on the clinical workflow which was discussed previously in section 2 three main variants are considered: 3 Viewing Concepts 8
• Pre-PACS Viewing
• Post-PACS Viewing
• Cloud Viewing
While all of those options work differently, they are not mutually exclusive with each other. This means that a viewing solution could, in theory, implement multiple ap- proaches at once. Throughout the following chapter, those three potential viewing types will be defined and analyzed. Additionally, their advantages and disadvantages resulting from their architecture and position in the pipeline will be exposed.
3.3.2 Pre-PACS Viewing
Figure 3: Pre-PACS Viewing
As the name suggests, this viewing type interferes right before the reconstructed data is sent to the clinical network (Figure 3). After the acquisition is finished and the scan is reconstructed, this type of viewing creates an option to locally view the scan. This may even enable the option to discard the scan without ever sending it to the PACS. As a result, there are multiple advantages and disadvantages that may be relevant for chosing the appropriate type of viewing solution. As the possibility of analysing a scan at an early stage in the scan pipeline allows for an early quality check of the image, mutiple useful improvements may result from this. On the one hand this would enable the purpose of early quality checking (section 3.2.2). On the other hand, this could also be used to reduce the PACS load (section 3.2.3). For both of those purposes, it is essential that the radiologist is able to view the scan as soon as possible to identify bad quality scans - so that he is able to create a better quality scan and/or discard the bad quality scan. This is addressed by this particular viewing type. 3 Viewing Concepts 9
Another aspect which could prove to be important is the availability of the correspond- ing raw data. When fetching the scan data directly from RWS after the reconstruction is done, the corresponding raw data is still available and easily identifiable. This data could, in theory, enable the radiologist to reconstruct the same scan with different parameters - without creating the need of acquiring another scan. Last but not least, even the reconstructed DICOM data and the RWS itself could be directly accessible due to the local proximity. Direct file and folder access to RWS would allow for the use of custom scripting solutions which could be used to further automate the functioning of the viewing solutions, e.g. monitoring for new scans to display. Even though all of those advantages could bring improvements to the clinical work, a solution of this kind comes not without limitations. The most obvious one would be, that the preview scan will only be available locally at one specific workstation. Viewing the scan on other devices or workstations would not be possible. Depending on the purpose of the viewing solution this could be a problem and has to be considered in before. Another factor which has to be taken into account when deciding on this viewing type are the hardware requirements. As the reconstructed scan data is previewed on a local workstation, this workstation needs to fulfill all necessary requirements for handling the rendering of the image. Those requirements are increasing, the higher the desired viewing quality is. If advanced viewing or maybe even post-processing capabilities are desired at this early stage in the workflow, this quickly evolves into a relevant factor. Especially three-dimensional rendering options are quickly pushing a lot of hardware to its limits.
3.3.3 Post-PACS Viewing
Figure 4: Post-PACS Viewing
Similar to the "Pre-PACS Viewing", this solution relies purely on local data and local rendering. The main difference here lies in the source of data which is used. Contrary to viewing the data right before sending it to the clinical network, a solution of this 3 Viewing Concepts 10 type queries the data from the PACS before displaying it, as displayed in Figure 4. In direct comparison, those solutions have multiple disadvantages. First of all, the reconstructed data has to be sent from the RWS to the PACS before viewing it. It then has to be retrieved again at the viewing workstation. This creates an unnessecary amounts of network load, but also costs time. The combination of components (RWS, Viewing, PACS) may also introduce new difficulties in associating the corresponding raw data with the scan. In addition the rendering is still executed locally. The hardware requirements remain, accompanied by a potential network bottleneck by sending/retrieving data to/from the PACS. This may lead to a user experience which varies greatly depending on the used hardware (client and server side). A good experience can only be guaranteed if the clinical network is able to handle the necessary amounts of data in reasonable amounts of time.
3.3.4 Cloud Viewing
Figure 5: Cloud-PACS Viewing
The third way of implementing viewing for a medical product would be a cloud solution. Viewing via cloud resources handles a lot of things differently than the local solutions which were discussed before. A typical cloud setup is shown in Figure 5. The character defining feature in those solutions is the independence of the end device. Most of the available zero-footprint viewers rely solely on HTML5 on client side 2. The end device uses a webbrowser to access the viewer interface. The requested DICOM study is then processed by an application server which is also responsible for rendering the images [2]. This is especially important as this removes near to all hardware requirements from the client devices. The application server then streams the rendered image to the target device for viewing. Due to the low hardware requirements, those end devices can range from small mobile devices to full-sized workstations. This improves the user experience greatly as no local software has to be used and the 2An example of typical client side requirements can be seen in [9, p. 12] 3 Viewing Concepts 11 user interface is unified independently of the end device. As great as this option is for viewing purposes on a large scale network of multiple end devices, an implementation of this type is out of scope for a single medical product. Though, if a solution like this is already existent in a customers hospital, it could be used for close-proximity viewing purposes as long as the medical device fulfills the required standards. Furthermore, cloud viewing has multiple disadvantages which are rendering it useless for specific purposes of viewing. The main problem in this context would be, that there is no direct access to the scan data. As the viewing client only displays an already rendered stream, no local data copy is available on the end device. This reduces the possibilities of implementing own features and limits the capabilites, which a solution of this kind could have. Especially in terms of automation and data exporting, a lot of products are very limited. As the raw data is not accessible as well, the possible scope for implementing own features is pretty small. Reconstructing the same scan with different parameters could be hard to realize.
3.4 Viewing Pipeline
3.4.1 Overview
For being able to define a set of desired features, considering the viewing solution as a whole is not beneficial. To many factors come into play in different stages of the viewing process. To simplify this, the viewing will be splitted in four different phases, which will be looked upon seperately, according to Figure 6. Then, based on the goal of each phase, an overview of all possibilites to realise this purpose will be created. More specifically, this means the most common features as well as the technical restrictions or necessities for realising each phase will be described. This allows to create an overview about every different viewing phase on its own. Together with the previously discussed viewing purpose and the viewing types, this pipeline will lay the foundation for deciding on the required feature set. Splitting up the viewing into multiple phases allows to easily select required features per phase, as each phase has a clearly defined function. Following on this chapter, which will give a broad overview regarding the different purposes, types and features of viewing solutions, the specific needs for a software will be worked out. This allows to finally decide on a solution for implementing a prototype.
3.4.2 Phase 1: Input
The first phase of the pipeline is the input phase. Regardless of the specific imple- mentation of the viewing solution, viewing always starts with loading some input data, 3 Viewing Concepts 12
Figure 6: Viewing Pipeline which should be displayed. They most important questions to consider for this part are:
• Where does the input data come from?
• Which options for accessing them exist?
As for the source of the input data, there are three options which have been described earlier already (section 3.3). The data can be obtained either from the RWS itself, from the PACS or as a pre-rendered stream from the cloud. Both options can be viable depending on the purpose. While the general difference of those options should be clear by now, the technical realisation options which exist for each of those sources have not been discussed yet. To get data from the RWS itself requires the transfer of potential big DICOM studies. While in theory those files could be opened and copied manually to the VWS, an automatic way would be much more convenient. One way of doing this could be folder monitoring of the reconstruction storage. As soon as a new scan is recognized, the copying and loading of the image data to the viewing solution is done automatically without any user interaction. If the data should be retrieved from the PACS, this could be achieved using the DICOM Query/Retrieve service [5] or the DICOMweb standard [8]. Those are the default ways of fetching studies in a DICOM conforming PACS.
3.4.3 Phase 2: Display
After the data is transferred to the viewing solution, one has to consider different displaying options. A lot of different options for displaying the data exist. In addition, there may be different projection methods which could be used, too. Depending on the purpose of the solution, the relevant ones have to be chosen. The key questions for this phase are:
• Which options for displaying the data exist?
• How is the data presented?
There are a lot of different options for displaying the data. Those options depend mostly on the selected viewing software. Although the variety here is pretty big a 3 Viewing Concepts 13 few features are very common and exist in nearly every solution. The most important one here would be the ability to view the reconstructed images from different angles or anatomical "planes". This is visualized in Figure 7 by using the reconstructed CT image of a Playmobil figure which was acquired using nu:view. One can imagine those planes by thinking of a human body which is standing in front of the viewer. The different planes are then seperating the body into different sections. The transversal plane is seperating the body into an upper (Superior) and lower (Inferior) section. The sagittal plane is seperating the body into a Left and Right section, and the coronal plane is seperating the body from front (Anterior) to back (Posterior) [3]. The reconstruction algorithm is outputting a transversal view by default. The "Multiplanar Reformation" or short "MPR", allows to create those coronal
(b) Coronal View (a) Human Body Planes Source: [3]
(c) Sagittal View (d) Transversal View
Figure 7: Multiplanar Reformation and sagittal views from the transversal slices, which were created by the reconstruction algorithm. This allows for evaluating the scan from different angles and is a default feature in lots of viewing solutions. Another interesting feature option which is available in some software, is the possibility of using different projection methods, e.g. maximum intensity projection. Maximum intensity projection creates 2D images from 3D reconstructions (or multiple 2D slices) and highlights the structures with the highest intensity. In addition, a lot of viewing 3 Viewing Concepts 14 options are offering 3D volume reconstructions which allow to watch the reconstructed scan data as a 3D model. Due to the nature of 3D rendering, this may require stronger hardware for fluid rendering.
3.4.4 Phase 3: Editing & Post-Processing
While displaying the reconstructed image data, the most viewing solutions offer a set of editing and processing options. Those features allow for manipulating the image data in different ways. In this context it is important to define:
• Which editing/processing options may be relevant for the specific use case?
• How are those options accessible?
The most common set of features are the basic image manipulation features. Zooming and rotating the image are possible in every software solution and are core features for every image viewing software. A lot of software solutions are also offering features for coloring or marking specific areas. Common tools for this are a pencil for free annotations, brightness and contrast sliders for enhancing image clarity and the option to measure distances in the image. While those basic options may fulfil a lot of purposes already, some more advanced features are offered by some programs. First of all, the selection of a region of interest (ROI) is offered in some tools. The most common way is to offer the user a rectangular tool to select areas. Other geometrical shapes can be possible, too. Often, the minimum and maximum intensities, as well as the mean intensity and the standard deviation are calculated for those selected areas. Some software solutions are even offering an automatic detection of contours for creating a region of interest which is not bound to any specific geometrical shapes. Most ROI options are two dimensional, which means they are based on a specific slice. However, more advanced tools even allow for selecting three dimensional VOIs based on multiple slices. To achieve this, a possible way is to define a cuboid by selecting a rectangular area in each MPR plane. The possible use cases of those regions and volumes of interest is broad. Quality assurance comes to mind, as the calculated mean or standard deviations are often used for verifying the correct functionality of the hardware by comparing them to reference values for specific test scans. Another possible use case could be the visual selection of specific areas for exporting their coordinate values for further use. Being able to clearly define specific image regions by their coordinates could allow for post-processing options for those limited areas.
3.4.5 Phase 4: Output
After displaying and editing the data is done by the user, relevant data may has to be exported for further use. This could range from simple textual information 3 Viewing Concepts 15
(e.g. coordinates) to new (and possibly edited/manipulated) image data. Questions to consider:
• Which data should be returned?
• How can this data be returned?
The most straight-forward option would be to save the manipulated image as a whole new set of images. This could happen as a new DICOM study, which contains all the manipulations/editings. This option is offered by most existing viewing softwares. Another option is to be able to export specific manipulated slices as single images. This allows for easy sharing of those images via email or similar. The export of manipulated slices to JPG or PNG file formats are the most common ways of doing this. On the other hand, it may be required to export specific ROIs which have been selected in the previous phase. Those ROIs (or VOIs) are represented by coordinate values. While a lot of software does not allow for an export of those values, few solutions are offering an export as text files. An additional way of outputting information is to display the current position of the mouse pointer as coordinate values. This can be useful for roughly defining the position of specific features which are visible in the image. Most viewing solutions are offering this possibility. 4 Requirements 16
4 Requirements
4.1 Retrospective
When deciding for a software to integrate into the existing pipeline it is essential to define its requirements exactly so that a sensible and fact-based comparison can be made. The goal of this chapter is to define an exact set of features that a viewing solution has to offer for fulfilling the specific needs for integrating it into the nu:view pipeline. The last sections intended to give a brief overview about the possibilites that exist for integrating a viewing solution. At first, different possible operational purposes were discussed, as all of them have to be considered when defining the specific use case that the chosen solution should fulfill. Following this, the most common types of viewing regarding the position in the clinical workflow were described. Those types are relevant as the selected type may limit the use cases which can be realised. After deciding on a use case, the best fitting viewing type should be specified which in return already limits the range of feasible viewing solutions that have to be considered. In the final step, the viewing workflow was splitted up into different phases. This allowed to further specifiy the goals and key questions that have to be considered in each step. In addition, a brief summary was given for each phase where the most common features were listed and explained. The total of this information regarding operational purposes, viewing types and viewing pipeline will now lay the foundation for exactly defining the requirements a viewing solution has to meet.
4.2 Desired Purposes
Due to the vast amount of viewing software and their different sets of features, not every software is capable of fulfilling every intended use case. The first step is to define the exact use case(s) the viewing solution for the nu:view system has to fulfill. As mentioned earlier (section 3.2) the use cases for a viewing solution may range from simple previewing software to extended diagnosis and editing software. While a lot of potential use cases were mentioned in customer feedback during the last years, it may not be feasible to pursue all of them. In the following section, the relevance of the different possible use cases will be evaluated more precisely and the most important ones will be defined. As part of the european medical device regulation, manufacturers of medical devices have to do a regular clinical evaluation after placing the product on the market [14, Article 10, Paragraph 3]. In the case of nu:view, a survey was part of the clinical evaluation. It was conducted at multiple hospitals where the nu:view system was in use. During the last years clinical use and installation tasks, Advanced Breast-CT 4 Requirements 17 also managed to get a lot of oral feedback from clinical staff members. Of special importance are features which were requested by people who are working with nuView on a daily basis.
4.2.1 Clinical Evaluation
The clinical evaluation survey was realised as a printed questionnaire which was filled out by hand by the survey participants. The group of survey participants consisted of four radiologists and five radiology technicans from different hospitals. All of the participants were working with nuView on a regular basis. The survey was done in 2020 and contained six questions. The questions were aimed at the topics of product appearance, product quality, workflow and patient comfort which the participant had to rate. For each question the participants were able to chose between five ratings: Perfect, Good, Moderately Good, Inadequate and Not Applicable. They also had the option to add additional comments to each question. While the overall results were positive, the survey gives some relevant information. The most indicative point are the results of the image quality ratings. More than half of the participants were not able to rate the image quality. Further research shows that only radiologists were able to rate the image quality while the participating MTRAs did not answer the question. This is very characterizing for the current workflow where the ability to view the scan is first given at the diagnosis workstation. As the radiologists are responsible for the diagnosis, the MTRAs who are acquiring the scan have no options to see it during their usual workflow. More useful information was given in the comments. Multiple participants claimed that "Reconstructed overview images were not provided". Another participant claimed that "Faster reconstructions in general would be desirable".
4.2.2 Other Feedback
Aside from the clinical evaluation which already gave some good information regarding potential problems in the viewing workflow, some additional feedback was received. In a telephone call, a radiologist claimed that scrolling through the CT images takes a lot of time, as a lot of slices contain only air as a result of the fixed scan lenghts. During a regular meeting, three radiologists requested to get an additional reconstruc- tion profile to get an overview for quality checking in a first step.
4.2.3 Conclusion
The entierety of this different kinds of customer feedback now allows to define two main problems:
• Reconstructed overview of images is not provided 4 Requirements 18
• Reconstruction could be faster
Those problems could be solved by a viewing solution which fulfills the following op- erational purposes:
• Preview the scan close to scanner
• Allow high resolution reconstruction for specific VOI
On the one hand, a preview close to the scanner eliminates the problem of the radiology technicans which are not able to see the results of their work. It would give them the desired reconstructed overview images and enable basic quality checks before sending the data to the clinical network. The possibility of high resolution reconstructions for a specific region of interest can speed up the reconstruction times as only selected parts of the image have to be reconstructed instead of a whole scan. This saves time and storage. Based on those purposes, a set of required features can be defined. Those features are necessary to fulfill the chosen purposes. For better separation, the requirements will be splitted into functional and non-functional requirements.
4.3 Non-Functional Requirements
4.3.1 Overview
The selected software solution has to fulfill some basic quality standards. Those are considered the "Non-Functional Requirements". They are necessary to secure a flawless and easy application in the clinical context.
• Clean UI: The most important functions should be easy to use. The UI should be self-explanatory and not too complex.
• Performance: As the reconstructed scans are still pretty big in size, the applica- tion has to deliver a good performance. The loading times should be reasonable for experimental use.
• Stability: The program should be able to handle the viewing and processing of big reconstructions without crashing or freezing due to high amounts of data.
4.4 Functional Requirements
4.4.1 Overview
While the non-functional requirements were pretty general, the functional requirements are very specific. They are necessary to allow the use of the software for the desired purposes. This section will discuss those features for each step of the viewing pipeline 4 Requirements 19
VOI Feature Preview Reconstruction Local DICOM Import x x CLI x MPR x x VOI Selection x VOI Export x
Table 1: Requirements for Different Purposes which was defined in the previous chapter. For each feature the reasoning why it is considered a necessity will be explained. Table 1 gives a brief overview about those requirements.
4.4.2 Input: Local DICOM Studies
The solution needs to be able to retrieve its data directly from the filesystem of the RWS. This is required for the preview as well as for the detailed reconstruction purpose. In the case of the preview this reduces the amount of time as the data does not have to be send to the PACS first and then be retrieved from there. In addition, less storage is used as it allows to discard bad quality scans instead of storing them in the PACS. For the detailed VOI reconstruction, this is an essential feature, too. After sending the reconstructed scan to the PACS, its raw data is not easily available anymore. Without access to the raw data, reconstructing is not possible. It is therefore necessary to receive the scan data directly from the filesystem of the RWS instead of the PACS.
4.4.3 Input: Command Line Interface
As the goal of this project is to create a prototype for a solution that can be used in a clinical context, it is important that the viewing software can be started via command line interface. There has to be a possibility to at least start the solution and load a DICOM study via command line interface (CLI), as this allows further automation later on - for example automatically starting the viewing when a new preview scan is recognized.
4.4.4 Display: MPR
To check for correct patient positioning or motion artifacts, it is necessary that the reconstructed scan can be viewed from different angles. The MPR feature allows this. Additionally, an MPR view allows for an easy selection of VOIs. While selecting a VOI in a "real" 3D view could be possible, the easiest and most precise way is to select the volume dimensions in MPR view. 4 Requirements 20
4.4.5 Editing: VOI Selection
The software needs to allow for a selection of a 3D volume of interest. This should happen in a cuboid shape as the reconstruction CLI is accepting cuboid shapes as reconstruction regions. More details on this will follow in section 5.1.4.
4.4.6 Output: Export VOI Coordinates
Especially for the detailed VOI reconstruction it is mandatory that the software allows to export the coordinates of selected VOIs. Without an export option, the VOIs can not be used for further purposes or reconstructions.
4.5 Comparison of Tools
4.5.1 Overview
To chose a software which will be used for implementing a prototype, a set of existing software solutions will be compared. This comparison will be done based on the defined requirements. For each software it will be checked which requirements are met. The software solutions that were chosen are representing a broad variety of viewing solutions. They contain open source solutions as well as commercial solutions. It has to be noted, that only software which allowed the import of local DICOM studies has been considered. As noted previously, cloud solutions are out of scope for this project and software which can only retrieve studies from a PACS is not usable as access to the scans raw data has to be possible for the VOI reconstructions. Additionally, our in- house developed viewing software ImpactView will be part of the comparison. Table 2 shows the results of this comparison. It states for each software, which requirements were met. For some of the more promising tools, a more detailed description of their features and limitations will be given afterwards.
Non-Func. VOI VOI Software CLI Local DICOM MPR Requirements Selection Export MicroDicom no yes yes no no(*) no RadiAnt DICOM yes yes yes yes no(*) no Aliza DICOM no no yes yes no(*) no Sante DICOM no yes yes yes no(*) no ImageJ no no yes no no no 3D Slicer no no yes yes yes yes Weasis yes yes yes no no(*) no ImpactView yes yes yes yes yes no
Table 2: Software Comparison (*) = 2D only 4 Requirements 21
4.5.2 RadiAnt DICOM Viewer
RadiAnt DICOM viewer is a commercial DICOM viewer developed by the polnish company medixant 3. It offers an easy to use and fast interface as well as command line options for loading scans and even changing its appearance. For example it allows for setting its window position, size and to hide sepcific toolbars. For the sole purpose of previewing scans it is a very good solution. However, it does not offer any options for exporting VOIs so it is not suitable for implementing a VOI reconstruction.
4.5.3 3DSlicer
The 3DSlicer 4 is an open source project for viewing and manipulating 3D volumes. It supports the editing and viewing of local DICOM studies. Multiple different views are offered, including MPR view. Additionally the software allows to select VOIs in MPR view and allows to export them as text file. While all those features seem to be very promising, the tool has some major disadvantages which render it useless for prototyping. First of all, it offers no easy to use CLI options for loading a DICOM study directly via command line. An automated preview would be hard to realise without this feature. A python interface is offered instead. While this interface is very capable in theory, its documentation is incomplete and complex. Secondly, and even more important is its interface. 3DSlicer offers a very big amount of features and a big part of them are not necessary for the use case which should be realised. This leads to a very complex interface as well as bad performance.
4.5.4 ImpactView
ImpactView is a viewing software developed by AB-CT. It has an easy to use UI and is able to load local DICOM files using its GUI or its command line interface. An MPR view is offered. It gives options for selecting regions and volumes of interest of different shapes, including a cuboid volume of interest. This VOI can be selected easily in the MPR view. This makes ImpactView a promising candidate. However, exporting the selected re- gions is not possible. While the user is able to see the coordinates of the selected areas in the UI, there is no export option to save them. The user would have to copy the coordinate values manually to make further use of the regions.
4.5.5 Conclusion
During this chapter, a multitude of different DICOM viewing softwares was tested. Each tool was compared to the requirements and the goal was to find a solution which 3For more information on RadiAnt DICOM Viewer use [13] 4For more information on 3DSlicer use [1] 4 Requirements 22 fulfills them all. This goal could not be reached. Nearly all tools allowed for viewing the scans and displaying them in a MPR view which made them usable for previewing. In contrast to this, there was no software which fulfilled all requirements which were necessary to make a detailed reconstruction of a specific area possible. Most software did not allow for selecting 3D volumes of interest. An exception was 3DSlicer. it allowed for selecting and even exporting volume coordinates. This makes it a valid option for prototyping a solution based on those volume coordinates. The problem is that 3DSlicer is a very feature rich tool which exceeds the needs for the defined use cases by far. It is pretty complex and while it would give the technical possibilites to realise a prototype, its performance and usability are lacking. For this reason ImpactView will be the first choice. ImpactView does offer all necessary tools to allow for an easy previewing as well as an easy VOI selection. Its biggest problem is the lack of an export feature for the selected regions. However, as ImpactView is an in-house devlopment of AB-CT, its source code is available. This gives the possibility to modify the software and add features based on the needs of this project. 5 Proof of Concept 23
5 Proof of Concept
5.1 Implementation
5.1.1 Overview
AB-CTs in-house developed viewing solution "ImpactView" was chosen to implement and integrate a proof of concept in hardware and software. The implementation of this protoype will be described in this chapter. The first step in creating a working proof of concept, will be the implementation and configuration of the software components on a single local workstation. Following this, in section 5.2, the setup will then be transferred an reconfigured for the nu:view test setup consisting of multiple workstations emulating the real clinical setup.
5.1.2 Pipeline
To efficiently prepare, configure and implement the software, the components have to be defined precisely. Additionally, the interaction between every component has to be clearly defined. For this reason, a pipeline was created. This pipeline represents the complete workflow from receiving a new preview scan to selecting a VOI and initiating a higher resolution reconstruction. It will be described in the following. The main goal here is to understand the general workflow. After the pipeline is created and described, a more in depth overview about the im- portant components will be given. For each component, their detailed implementation will be shown and it will be described why their specific type of implementation was chosen. Figure 8 illustrates the basic workflow which the protoype should realise. To create
Figure 8: Basic Pipeline the possibility of an automated preview the first important step is to recognize a new reconstructed scan. The viewing workstation has to automatically recognize when a new scan for the preview arrives. In the current workflow each scan is represented by a folder of a unique name. Based on those preconditions, a simple folder monitoring is sufficient. The "ScanMon- itor" should permanently monitor a chosen folder where the preview scans are stored. When a new folder is created, an event should be triggered, which loads ImpactView for previewing the scan and for selecting the VOIs. Loading ImpactView itself is a 5 Proof of Concept 24 pretty easy step. It supports loading image data via command line arguments. A di- rectory contaning DICOM images can be passed to the software just by adding its path as a positional argument. ImpactView then loads the image data directly at startup without any further interaction needed by the user. After the scan is loaded and viewable in ImpactView the detailed VOI reconstruction is the next step to realise. ImpactView offers an option to easily select cuboid VOIs in a volume. This is achieved by selecting multiple rectangular areas in the three different MPR planes. The coordinates for those areas are shown in ImpactViews UI, as visible in Figure 9. However, there is no option to export them. One important step in the realisation of the pipeline is therefore to find a way to retrieve those VOI coordinates from ImpactView in the easiest way possible. They can then be used to start a new, higher resolution reconstruction for the selected area. This new reconstruction is the last step in the pipeline. The actual reconstruction of the raw data is handled by ImpactReconCLI. ImpactRecon- CLI is a command line application which takes multiple inputs and parameters to re- construct a volume from raw data. To realise the reconstruction of a limited VOI, multiple parameters have to be known:
Figure 9: Example Selection of VOI
• Path to raw data: As the user is selecting the VOI in a reconstructed version of the scan, the path to its raw data has to be known. Else, a higher resolution reconstruction of the scan data is not possible.
• VOI Coordinates: The region which should be reconstructed in higher resolution has to be defined exactly. ImpactReconCLI offers options to pass specific recon- struction regions as optional arguments. The exact way in which those regions 5 Proof of Concept 25
have to be represented for using them with ImpactReconCLI will be described later.
• Reconstruction Quality: The quality setting which should be used for the re- construction. There are multiple presets availble, which are mainly differing in resolution and type of filter.
When all those parameters are known and in the correct format, ImpactReconCLI is able to reconstruct a specific area of the raw data in a previously defined quality. This represents the last step and the final goal of the pipeline. The next chapters will now describe the implementation of those components in detail. This chapter intended to give an overview about the general functionality of each pipeline component. In addition it was roughly pointed out which data or information has to be transferred between those components. Following on this, the next sections will give a more in-depth description of the implementation of those important parts. The chosen approaches will be described and implementation examples will be given for the most relevant features.
5.1.3 Scan Monitoring
As described in the previous section, some sort of scan monitoring for recognizing new scans had to be implemented. Because PowerShell offers relatively easy options for folder and file monitoring, it was used to create an example implementation 5.
1 $watcher = New-Object System.IO.FileSystemWatcher
2 $watcher.Path ="D:\ReconData"
3 $watcher.Filter ="*.*"
4 $watcher.IncludeSubdirectories = $false
5 $watcher.EnableRaisingEvents = $true
Listing 1: Creation of FileSystemWatcher
Listing 1 defines a FileSystemWatcher objekt which is used to monitor a defined path with specific settings. It raises events for different types of changes to the watched directory. The next step is to define an action, which should be triggered when a new scan is recognized. For this application, this means when a new folder is created in the monitored directory.
1 $action = {
2 $path = $Event.SourceEventArgs.FullPath
3 # Check if folder or file:
4 if((Test-Path -Path $path -PathType Container)){
5 # Check if copying is finished:
6 $LastLength = (Get-ChildItem $path).length
7 Start-Sleep -s5
5Use official Powershell documentation for reference [11] 5 Proof of Concept 26
8 $NewLength = (Get-ChildItem $path).length
9 while ($NewLength -ne $LastLength){
10 $LastLength = $NewLength
11 $NewLength = (Get-ChildItem $path).length
12 }
13 # Load preview
14 $impactViewPath ="Path\to\ImpactView"
15 Start-Process -FilePath ($impactViewPath +"\ImpactView.exe")- ArgumentList $path
Listing 2: Example Realization of Monitoring
This action (Listing 2) achieves multiple things. It checks the path where the event is raised from. If the path is a folder, it needs to make sure that the scan transfer is already finished before opening the scan with ImpactView. To achieve this, a size comparison is made every 5 seconds. If the folder size did not change, the script assumes that the transfer is complete. ImpactView will be started and the scan will be loaded. The last step is to connect an event to this action. The following code connects the "Created" event, which triggers on creation of files and folders, to our action.
1 Register-ObjectEvent $watcher"Created" -Action $action
Listing 3: Connecting an Action to an Event
This script can now be run in background in an infinite loop to achieve an ongoing folder monitoring.
5.1.4 ImpactReconCLI
The ImpactReconCLI is the unit handling the actual reconstructions. As the name suggests, ImpactReconCLI is a command line tool with no GUI. For the goal of re- constructing a specific VOI, the program has to be called with specific arguments and parameters. It offers two interfaces for this purpose: The command line interface which controls the program itself, and the parameter file interface which controls the reconstruction algorithm. The following arguments and parameters had to be modified for the pur- pose of a VOI reconstruction. Command Line Interface:
• –inputpath: Defines the path of the raw data.
• –profile: Defines which reconstruction profile to use. This also defines the size of each voxel (String).
• –output: Output directory of reconstructed scan (String, optional). 5 Proof of Concept 27
• –outputname: Name of the reconstruction (String, optional).
Parameter File Interface:
• Orientation: Defines the orientation of the resulting DICOM value (Enumera- tion).
• Origin: Defines the real-world position of the voxel-center (3D-vector of floats).
• NumXVoxels: Defines the number of voxels in x-direction (Integer).
• NumYVoxels: Defines the number of voxels in y-direction (Integer).
• NumZVoxels: Defines the number of voxels in z-direction (Integer).
An example batch call which starts a VOI reconstruction could look like Listing 4:
1 set PROFILE=HighRes
2 "%IMPACTRECON%"
3 --profile %PROFILE% -i !INPUTPATH!\scan_params.cpv
4 %dicomOrientation% RIA
5 % Origin %"(%X_ORIGIN%,%Y_ORIGIN%,%Z_ORIGIN%)"
6 %Nx% %X_VCOUNT% %Ny% %Y_VCOUNT% %Nz% %Z_VCOUNT%
7 -o %OUTPUT% -on %INPUTNAME%
Listing 4: Reconstruction Example
It is important to notice that shortcuts are used for easier modifications to the pa- rameter file interface. This is achieved by parsing a shortcut textfile which maps the variables used in the above call to the exact command line argument which is used to change those parameters. For easier understanding, some examples of those shortcut mappings are given in Listing 5.
1 Origin=--modifyparam setparam /Volume/Origin
2 Nx=--modifyparam setparam /Volume/NumXVoxels
3 Ny=--modifyparam setparam /Volume/NumYVoxels
4 Nz=--modifyparam setparam /Volume/NumZVoxels
Listing 5: Shortcuts Example
Those shortcuts are parsed line by line and used as variables, by setting them in a FOR loop Listing 6:
1 for /f"delims=" %%x in (shortcuts.txt) do ( set"%%x")
Listing 6: Batch Parsing Example
This allows us to set those values easily. An example is shown in Listing 4. %Nx% %X_COUNT_STD% sets the parameter /Volume/NumXVoxels to the value of X_VCOUNT by using the –modifyparam option as seen in Listing 5. 5 Proof of Concept 28
5.1.5 Coordinate Extraction
The core problem when creating a detailed reconstruction is the extraction of the co- ordinates. When the user selects a region in the reconstructed volume, the selected area has to be extracted for use with ImpactReconCLI. ImpactView in its current state does not offer this option. Although exporting is not possible, ImpactView does offer the possibility to see the coordinate values of the selected areas. To achieve this, the position coordinates and side lengths of the selected cuboid are displayed. The position always describes the corner to the top left front of the selected cuboid. The side lengths are describing the length of each side in mm. Based on this, a first approach to realise a working prototype was implemented. This first apporach consisted of a small GUI for the ImpactReconCLI. This goal of the GUI was to simplify the usage of the reconstruction. In detail, this means that it should provide options for all parameters and settings which are necessary to start a detailed VOI reconstruction. This simplifies experimenting with different VOIs, as no manual parameter modifications have to be done. To achieve this the core features of the GUI have to represent the parameters for ImpactReconCLI which were described before. The source code for the GUI was written in C++11 using the QT Framework 6. This
Figure 10: Basic VOI Reconstruction GUI apporach was proven to be functional. When a new scan arrives, ScanMonitor is able to load the preview and start the GUI automatically. The user then selects a VOI in ImpactView. Following on this, the user has to manually copy the coordinates of the VOI into the corresponding QSpinBoxes. After selecting a quality, the user is then able to start the new reconstruction by pressing the "Start" button. No command line interaction is needed during this workflow.
6For additional information about the QT framework and its classes refer to [12] 5 Proof of Concept 29
While this is a major step forward and already a working way to realise the VOI re- construction, it is still far from optimal. Copying the coordinates from ImpactView to the GUI takes a lot of manual work and is relatively time consuming. An important point to know is, that ImpactReconCLI expects the voxelcount for each side of the VOI bounding box as a parameter. ImpactView however, is displaying the side length of the bounding box in mm. Additionally, it has to be known that Im- pactView is by default using an "RIA" orientation for its coordinate system (Figure 11). The orientation of a coordinate system defines the order and direction (compare sec- tion 3.4.3 in which its axes are defined. As the voxelcounts are not necessarily equal to
left to right RIA = superior to inferior posterior to anterior
Figure 11: RIA Orientation the side length in mm, the side length which is extracted from ImpactView has to be converted to a voxelcount for the new resolution before starting the VOI reconstruc- tion. Essentially, this means dividing the side length from ImpactView by the voxelsize for the new reconstruction. The following voxelsizes are used by the different quality presets:
• Preview: 1mm
• Standard: 0.15mm
• HighRes: 0.05mm
A much more flexible option for further experiments, would therefore be to extract all necessary values into an easy parsable file. This exported file should contain all relevant information to reconstruct the VOI area in all quality settings. This leads to the second approach: An export option, directly integrated into Im- pactView. A GUI for manually entering the coordinates would not be necessary any- more as they could be easily parsed from the exported file. To realise an implementation of this feature, it is important to know, where this information is stored. As shown in Figure 9, the coordinates for the selected area are shown in the right half of the UI. This right half is the LayerManager. which is represented by an own class. As a detailed class description for this class is not necessary to understand the following implementation, it was deliberately omitted. The important thing to know is, that the LayerManager is creating all its UI elements (e.g."Position" and "Seiten" labels and the corresponding boxes) during runtime based on the currently selected tool from the toolbar on the left. The VolumeManager class is handling the loaded volumes respectively the image datasets. The OverlayManager class is responsible for handling overlays - like the 5 Proof of Concept 30
cuboid VOI selection. The relevant attributes which are used to define a selected VOI sufficiently are:
• Position of Origin (0,0,0) in mm
• Number of Voxels of the selected bounding box in each (x,y,z) direction
• Size of a single voxel in mm
An easy implementation to realise would therefore be an "Export" button directly inte- grated into the LayerManager when showing the cuboid VOI selection. All necessary attributes can be accessed inside of this class. For adding the button, default QT syntax is used. The button has to be created and added to the LayerManager widget. Then, the buttons clicked() signal is connected to a public slot method which manages the behavior of the button.
1 #include
2 void onExportButtonClicked();
Listing 7: Necessary Additions to LayerManager.h
1 QPushButton* exportButton = new QPushButton("Export");
2 pLayout->addWidget(exportButton);
3 connect(exportButton, SIGNAL(clicked()), this,SLOT( onExportButtonClicked()));
Listing 8: Creating and Connecting an Export Button
The code in Listing 7 and Listing 8 defines an export action, creates an export button in the GUI, and connects the button to the action. Following this, the onExportBut- tonClicked() action has to be implemented aus in Listing 9 , Listing 10 and Listing 11.
1 // Define necessary pointers and variables:
2 VolumeManager* pVolumeManager = m_pLayerModel->getVolumeManager();
3 VSlicerVolume* pVol = pVolumeManager->GetVolumeInFront();
4 Overlay* pOverlay = getCurrentOverlay();
5 QString volName = pVol->GetName();
6
7 // Get values of selectedVOI:
8 CVector3d
9 CVector3d
10 CVector3d
Listing 9: Extraction of VOI Values 5 Proof of Concept 31
The code in Listing 9 shows an example of retrieving the relevant values of the cuboid VOI which is currently selected by the user. It is then necessary to calculate the voxelcounts for all possible target resolutions, as shown in Listing 10.
1 // Voxelcounts in all resolutions:
2 CVector3d
3 CVector3d
4 CVector3d
5 CVector3d
Listing 10: Calculation of Voxelcounts for each Resolution
The final step is to export those values to a file. For experimental purposes a QFileDi- alog was used, which allowed to easily export multiple different VOIs and save them with user defined file names. This simplified testing with different volumes or regions and is shown in Listing 11
1 // Select export path:
2 QString dir;
3 QFileDialog saveDialog;
4 saveDialog.setDefaultSuffix("txt");
5 dir = saveDialog.getSaveFileName(this, tr("Select File"),
6 QDir::currentPath(),"Text(*.txt)");
7 std::ofstream output_roi(dir.toStdString());
8
9 // Example for saving coordinate values to file:
10 output_roi <<"INPUTNAME=" << volName << std::endl;
11
12 output_roi << std::fixed << std::setprecision(4)
13 <<"X_ORIGIN=" << pos[0]
14 <<"\nY_ORIGIN=" << pos[1]
15 <<"\nZ_ORIGIN=" << pos[2]
16 << std::endl;
17 //...repeat for all positions, edges, sizes and voxelcount values.
18
19 output_roi.close();
Listing 11: Save VOI Values as Textfile
The values are written to the exported file line by line, in an easily parsable format.
5.2 Integration
For the purpose of an easy implementation, all pipeline steps were simulated on a single workstation computer. However, after the basic implementation is done, the pipeline 5 Proof of Concept 32 should be integrated into a test environment which simulates the real setup more ap- propriate. This setup consists of all workstations which are used in the default clinical setup. For a representative test run this means a Scanner (or Scanner Simulator), AWS, and RWS are used as Hardware. The AWS is running ScanCommander for controlling the Scanner, the RWS is running the ReconServer software for managing the reconstruc- tions. In this case, this means specifically the full initial reconstruction which will then be previewed on the VWS. The VWS runs the modified ImpactView version and the folder monitoring for recognizing new scans for previewing.
5.2.1 Hardware
The workstations which were used for testing were configured according to Table 3 and Table 4. This represents the default configuration for AWS and RWS, as they
OS Windows 10 Enterprise OS Windows 10 Enterprise CPU Intel Xeon E5 2620v4 CPU 2x Intel Xeon Silver 4110 GPU Nvidia Quadro P400 GPU 3x Nvidia Quadro P5000 RAM 128GB DDR4 RAM 128GB DDR4 SSD 1024GB SSD 1024GB HDD - HDD 6TB als RAID6
Table 3: AWS Hardware Table 4: RWS Hardware are shipped to the customer. In contrast, the hardware requirements for the VWS are pretty low. Only reconstructed scans, which are much smaller than the raw scans have to be transferred and no reconstruction is performed on the workstation itself. The main requirement is a CUDA capable graphics card. This is necessary to guarantee a fluid viewing experience in ImpactView. However, the hardware configuration which was used for testing was identical to the RWS hardware. This was purely based on the fact that it was the only available workstation at the time of testing.
5.2.2 Network Topology
All of those different workstations and hardware are connected via multiple different interfaces. The default network setup of nuView includes the following devices:
• nu:view Scanner
• AWS
• RWS
• PACS/Clinical Network 5 Proof of Concept 33
In addition to the default setup, the VWS is added for the purpose of this test. This leads to the following network layout. For more information on the default workflow
Figure 12: Network Layout refer to section 2.3. The most important thing when integrating the VWS into the existing setup, is the connection to the RWS. As the VWS will get its reconstructed preview data from the RWS, they have to be on the same subnet. In this specific setup, the 192.168.8.0/24 subnet was used and the connection was realised via a default layer 2 switch using 1Gbps ethernet.
5.2.3 AWS Configuration
The configuration of the AWS is pretty straightforward compared to the default set- tings. The only important thing here, is that the raw scan data has to be available for the VOI reconstruction. As the VOI reconstruction in its prototype works via a simple CLI call, it’s enough if the ScanStorage is shared via network. For the user who is executing the VOI reconstruction on the RWS, read access is sufficient as no changes to the raw data have to be made.
5.2.4 VWS Configuration
The VWS is the workstation where the automated preview should happen. This means that the modified ImpactView software has to be available. As ImpactView should be called automatically if a new preview scan is arriving, the ScanMonitor (as implemented in section 5.1.3) has to be configured and running. For this reason, multiple options have to be configured correctly. First of all, some sort of preview storage is needed. This "ViewingStorage" needs to be accessible from both the RWS and the VWS itself. To fulfill those requirements, a shared folder was created. Write and read access were given to the default VWS user. Then, the ScanMonitor has to be configured to monitor this path as in Listing 1 line 2. On the VWS, ScanMonitoring should happen automatically if a user is logged in. For this reason, a scheduled task is set up. 5 Proof of Concept 34
1 $trigger = New-ScheduledTaskTrigger -AtLogOn
2 $action = New-ScheduledTaskAction -Execute"Powershell.exe" -Argument "-WindowStyle Hidden-File Path\to\scanMonitor.ps1"
3
4 Register-ScheduledTask -Action $action -Trigger $trigger -TaskName" ScanMonitor" -Description"Monitor for new incoming scans in Viewing Storage"
Listing 12: Setting Up a scheduled Task
This powershell script (Listing 12) runs the ScanMonitor script in the background after a user logs in. If a new scan arrives at the ViewingStorage, ImpactView is executed and loads the new scan.
5.2.5 RWS Configuration
For the RWS, multiple things have to be considered. First of all, the batch files for parsing the coordinates and starting the VOI reconstruction have to be copied to the workstation. Also some modifications to the scripts have to be done. Additionally, the input data path for the VOI reconstruction has to be configured. This needs to match the previously shared ScanStorage path, so that the raw data can be accessed and reconstructed again. ReconServer allows for exporting reconstructed scans via context menu. A copy of the reconstruction is then created at this location. The configuration of the available export paths is done via XML files. Additionally, the login credentials for this shared storage have to be saved. This is done via PowerShell 7. It is very important that the configured has read and write access to the ViewingStorage, as it will be used to export the scans. Of course, the exported VOI text file contaning the coordinates has to be accessible, too.
7For reference: see "cmdkey" command in [11] 6 Evaluation 35
6 Evaluation
6.1 General
The implementation and integration of the prototype was done according to section 5. For testing purposes the ScannerSimulator was used, as the nu:view device was under maintenance. However, this simulation is sufficient for testing the pipeline and should be equivalent to the real device. The goal was to simulate a scan using the ScanCommander on the AWS and recon- structing it in standard quality on the RWS. It should then be exported to the previ- ously defined viewing storage and the preview should open automatically. Following on this, a VOI was selected in the preview and saved to a previously configured path. This VOI was then reconstructed by using the high resolution reconstruction profile. The VOI reconstruction was initiated manually by executing a batch file. To evaluate the quality of the created pipeline, it will be rated in six different cate- gories of quality characteristics. Those categories are described in [4, Chapter 9] based on ISO/IEC 9126-1. According to him, ISO/IEC 9126-1 defines those categories as follows: • Functionality: The functionality and behavior the software is supposed to de- liver [4, pg.109].
• Reliability: The capability of the software to to deliver a specified performance level [4, pg.124].
• Usability: The capability of the software to be used and easily learnable by the users [4, pg.130].
• Efficiency: The capability of the software to deliver reasonable performance considering the used resources [4, pg.128].
• Maintainability: The capability of the software to be changeable [4, pg.116].
• Portability: The capability of the software to be transferred to another envi- ronment [4, pg.132]. The pipeline will be evaluated for each of those categories. Only the subcategories (see [4, pg.110]) which were considered the most important will be discussed. For each category, the solution will be rated Good, Neutral or Poor.
6.2 Software Quality Measurements
6.2.1 Functionality
While there were minor problems at first due to configuration mistakes, the pipeline worked correctly after the setup was fixed. It delivered a quick automated preview of 6 Evaluation 36 the whole scan and the reconstruction of the selected VOI delivered a high resolution image of the correct area. Additionally, the same workflow was tested for a low quality "Preview" reconstruction profile. As the preview scan profile is available for experimental purposes only, this reconstruction had to be done and transferred manually instead of using the ScanCom- mander and ReconServer export. Similar to the first test, a VOI reconstruction was initiated afterwards. A set of example images of those results is shown in Figure 13. It contains a full preview, a standard VOI reconstruction and a high resolution VOI reconstruction. The software is behaving like expected. The different resolutions are
Figure 13: Example of VOI Reconstructions in Different Resolutions. clearly distinguishable. Thus, the functionality is rated "Good".
6.2.2 Reliability
The pipeline only works under specific circumstances. The folder names of the raw data have to be identical to the folder names of the reconstructed data. If the raw data is missing or placed at the wrong location, the VOI reconstruction can not be done. Their names have to be unique, as the folder monitoring relies on their uniqueness. Additionally, the VOI values have to be correct and valid. The VOI reconstruction will fail if this is not the case. However, the behavior of the preview as well as the VOI reconstruction is non-destructive. No changes are made to the raw data or the initial reconstructed images. This means that some kind of recoverability is given in the case of errors. For this reason, the reliability is rated "Neutral".
6.2.3 Usability
The software is an experimental project. The scan preview is easy to use, as it is fully automated. No user interaction is needed, except from exporting the scan to the ViewingStorage. For the VOI reconstruction, some interaction is needed though. The modified ImpactView GUI allows to easily experiment with different VOIs and export them for reconstruction purposes. 6 Evaluation 37
A problem is that the name and path have to be specified manually. If the name or path is wrong, the VOI reconstruction will not work. This is a major learnability and operability problem for users without any technical knowledge. The reconstruction itself is started by running a batch script. While this does not require any user in- teraction after its initial setup (except from running it), the attractiveness is lacking. However, it has to be kept in mind that this software is only meant to be used for experimental purposes by qualified engineers. For this reason, the usability is rated "Neutral" despite the mentioned problems.
6.2.4 Efficiency
As the main goal of the VOI reconstruction was to improve the speed of the recon- struction, some time measurements have been done. It is important to note, that the time which is required for the reconstruction is highly dependent on the size of the selected VOI. For measuring the reconstruction time and size, the follwing VOI sizes were used:
• VOI1: 42mm x 42mm x 42mm
• VOI2: 85mm x 85mm x 85mm
The results of the time measurements are shown in Figure 14. The full reconstruction size was 200x200x90mm. The initialization of the reconstruction parameters took 40 secondds. One can see that the reconstruction times for the VOIs are noticeably shorter
Figure 14: Reconstruction times for different VOIs than the reconstruction times for the full image. This is a clearly visible improvement. As the size of the selected VOIs is highly dependent on the use case, it is not suitable to calculate percentual values of improvement. The reconstruction algorithm itself is 6 Evaluation 38 complex and does not scale linearly with the volume size. The important observations at this point are, that the reconstruction times can be improved by selecting smaller regions. This is visible in the data. The full reconstruction is taking much longer than the reconstruction of VOI1. The reconstruction of the bigger VOI2 takes longer than the reconstruction of VOI1. The file sizes of the reconstructed images behaved in a similar way, which is shown in Figure 15. As the improvements can be pretty big, depending
Figure 15: Reconstruction sizes for different VOIs on the size of the selected volume, the efficiency of the solution is rated "Good".
6.2.5 Maintainability
A big part in creating the pipeline was to combine different existing components into a single pipeline. Each component (ScanMonitor, ImpactView, ImpactReconCLI) relies on the output of another component. If one component changes, the following compo- nents have to be modified, too. The changeability is lacking. If there are errors during the interaction of the different components, they can be hard to find as no error handling was implemented. Though the existing components which were used offered some basic error handling, this was not be part of this evaluation as they were mostly considered as "black boxes" during this project. As a result, the maintainability is rated "Poor".
6.2.6 Portability
Transferring the pipeline to another environment requires the modification of multiple components. This setup process is described in section 5.2. Even though none of this 6 Evaluation 39 modifcations is very complex, the installability is average given the experimental pur- pose of this project. A positive aspect is, that the exported VOIs are complete and contain all necessary information to start a reconstruction. They can be transferred between different sys- tems, as long as the necessary raw data is existing on the target machine and the setup was done correctly. A VOI always contains the original voxelsize and voxelcount, which would in theory even allow to reconstruct in new resolutions which are not yet implemented. However, the overall portability is still "Poor", as a lot of easy but time consuming configuration steps have to be done.
6.3 Conclusion
As the implementation and integration have shown, the pure functionality, as well as the efficiency of the prototype is good. Two of the compared software solutions have proved to be adequate solutions for preview purposes, if the use cases and requirements of the preview are limited to the earlier (section 4.2) defined aspects. Those two were RadiAnt DICOM Viewer and the inhouse developed ImpactView. When taking into account the second goal of the prototype, the detailed VOI recon- struction of specific areas, no third party software was able to fulfill the requirements. The only software which at least offered the export of selected VOI regions as coordi- nates was 3DSlicer. Still, 3DSlicer had a lot of problems in performance and usability which made it unnessecary complex to work with it. For creating a usable prototype the modified ImpactView version was the best way to realise this experimental setup. Still a lot of own implementation and software had to be done. In its current state the implementation purely is a proof of concept, but it is working as intended. The workflow is usable for experimental purposes and the reconstructed VOIs are looking as intended. They are improving the reconstruction speed and reducing the file sizes noticeably. A real world implementation for clinical use would be a very complex and therefore costly project. Especially the integration of the VOI reconstructions into the existing end user software solutions (e.g. ScanCommander and ReconServer) would be a huge task on its own, while options for transferring those VOI reconstructions back to the clinical PACS were not even considered at all. Those topics are essential when devel- oping a solution for clinical use but would go way beyond the scope of this thesis. As a final conclusion, it can be can said that this proof of concept was a success. Differ- ent options have been investigated and a way to achieve the desired functionality was found and realised. The evaluation has shown that a lot of future improvements can be made, especially regarding maintainability and portability. Nonetheless, development and planning of a "real world" solution for clinical use could profit from the gathered 6 Evaluation 40 information and different approaches. References 41
References
[1] 3D Slicer. 2020. url: https://www.slicer.org/ (visited on 10/25/2020).
[2] A Closer Look at SERVER-SIDE RENDERING - Technology Overview. Fujifilm. 2016. url: http://www.timed.pe/pdf/informatica/CatalogoSynapse5Fujifilm. pdf (visited on 10/22/2020).
[3] Anatomical Terminology. National Cancer Institute. url: https://training. seer.cancer.gov/anatomy/body/terminology.html (visited on 10/20/2020).
[4] Helmut Balzert and Peter Liggesmeyer. Lehrbuch der Softwaretechnik: Entwurf, Implementierung, Installation und Betrieb. 3. Aufl. Lehrbücher der Informatik. Heidelberg: Spektrum Akademischer Verlag, 2011. isbn: 978-3-8274-1706-0.
[5] C Query/Retrieve Service Class. National Electrical Manufacturers Association. 2020. url: http://dicom.nema.org/medical/dicom/current/output/chtml/ part04/chapter_C.html (visited on 10/20/2020).
[6] Robert H. Chopling, Johannes M. Boehme II, and C. Douglas Maynard. “PACS Mini Refresher Course”. In: RadioGraphics (1992). url: https://pubs.rsna. org/doi/pdf/10.1148/radiographics.12.1.1734458.
[7] DICOM Part 1: Introduction and Overview. National Electrical Manufacturers Association. 2020. url: http://dicom.nema.org/medical/dicom/current/ output/pdf/part01.pdf (visited on 10/20/2020).
[8] DICOMweb. National Electrical Manufacturers Association. url: https://www. dicomstandard.org/dicomweb (visited on 10/20/2020).
[9] MedDream DICOM Viewer User Manual. Softneta. 2020. url: https://www. softneta.com/files/meddreamviewer/doc/MedDream-DICOM-Viewer-User- Manual.pdf (visited on 10/22/2020).
[10] Oleg S. Pianykh. Digital Imaging and Communications in Medicine (DICOM): A Practical Introduction and Survival Guide. Berlin, Heidelberg: Springer-Verlag, 2008. isbn: 978-3-540-74570-9.
[11] PowerShell Documentation. Microsoft. url: https://docs.microsoft.com/en- us/powershell/ (visited on 10/20/2020).
[12] Qt Documentation. The Qt Company Ltd. 2019. url: https://doc.qt.io/ (visited on 10/20/2020).
[13] RadiAnt DICOM Viewer. Medixant. 2020. url: https://www.radiantviewer. com/en/ (visited on 10/25/2020).
[14] REGULATION (EU) 2017/745 OF THE EUROPEAN PARLIAMENT AND OF THE COUNCIL of 5 April 2017 on medical devices, amending Directive 2001/83/EC, Regulation (EC) No 178/2002 and Regulation (EC) No 1223/2009 and repealing Council Directives 90/385/EEC and 93/42/EEC. EU, 2017. url: https://eur- lex.europa.eu/legal- content/EN/TXT/PDF/?uri=CELEX: 32017R0745&from=EN.