Multiple Sclerosis Lesion Segmentation using Improved Convolutional Neural Networks Erol Kazancli1,2, Vesna Prchkovska3, Paulo Rodrigues3, Pablo Villoslada4 and Laura Igual1 1Department of Mathematics and Computer Science, Universitat de Barcelona, Gran Via de les Corts Catalanes, 585, 08007 Barcelona, Spain 2DAMA-UPC, Universitat Politecnica` de Catalunya, C. Jordi Girona, 1-3, 08034 Barcelona, Spain 3Mint Labs Inc. 241 A Street Suite 300, 02210 Boston, MA, U.S.A. 4Center of Neuroimmunology, Institut d’Investigacions Biomediques August Pi Sunyer (IDIBAPS), Villarroel, 170, 08036 Barcelona, Spain Keywords: Multiple Sclerosis Lesion Segmentation, Deep Learning, Convolutional Neural Networks. Abstract: The Multiple Sclerosis (MS) lesion segmentation is critical for the diagnosis, treatment and follow-up of the MS patients. Nowadays, the MS lesion segmentation in Magnetic Resonance Image (MRI) is a time- consuming manual process carried out by medical experts, which is subject to intra- and inter- expert variabi- lity. Machine learning methods including Deep Learning has been applied to this problem, obtaining solutions that outperformed other conventional automatic methods. Deep Learning methods have especially turned out to be promising, attaining human expert performance levels. Our aim is to develop a fully automatic met- hod that will help experts in their task and reduce the necessary time and effort in the process. In this paper, we propose a new approach based on Convolutional Neural Networks (CNN) to the MS lesion segmentation problem. We study different CNN approaches and compare their segmentation performance. We obtain an average dice score of 57.5% and a true positive rate of 59.7% for a real dataset of 59 patients with a specific CNN approach, outperforming the other CNN approaches and a commonly used automatic tool for MS lesion segmentation. 1 INTRODUCTION disease. Manual segmentation of MS lesions by ex- perts is the most commonly used technique and is still Multiple Sclerosis (MS) is a chronic neurological di- considered to produce the most accurate results alt- sease that afflicts especially the young population be- hough it suffers from many complications. First of tween the ages 20 and 50. It affects 2.3 million pe- all, it is subject to intra- and inter- expert variability, ople worldwide and can cause symptoms such as loss which means there are significant differences between of vision, loss of balance, fatigue, memory and con- two segmentations performed by two different experts centration problems, among others. It remains a very (due to slightly varying definitions) or by the same challenging disease to diagnose and treat, due to its expert at different times (due to fatigue or similar fac- variability in its clinical expression (National MS So- tors). Secondly there is a shortage of adequately trai- ciety, 2017). MS is characterized by lesions throug- ned experts given the huge amount of segmentation hout the brain that are caused by the loss of mye- need. Thirdly, the segmentation task requires valuable lin sheath around neurons in the brain, which is also expert time and concentration, which could ideally be known as demyelination. The lesions are visible in se- dedicated to other tasks. These drawbacks make it veral modalities of Magnetic Resonance Image (MRI) necessary and desirable to develop a semi-automatic with different contrasts. The number and the total vo- segmentation method that would assist experts in the lume of MS lesions are indicative of the disease stage task with a reduced amount of time and intra- inter- and are used to track disease progression. expert variability or, in the ideal case, a fully auto- The accurate segmentation of lesions in MRI is matic segmentation method which would obviate the important for the accurate diagnosis, adequate treat- need for experts and produce accurate/reproducible ment development and patient follow-up of the MS results. 260 Kazancli, E., Prchkovska, V., Rodrigues, P., Villoslada, P. and Igual, L. Multiple Sclerosis Lesion Segmentation using Improved Convolutional Neural Networks. DOI: 10.5220/0006540902600269 In Proceedings of the 13th International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications (VISIGRAPP 2018) - Volume 4: VISAPP, pages 260-269 ISBN: 978-989-758-290-5 Copyright © 2018 by SCITEPRESS – Science and Technology Publications, Lda. All rights reserved Multiple Sclerosis Lesion Segmentation using Improved Convolutional Neural Networks Several methods previously presented in the lite- segmentation of the whole brain (Greenspan et al., rature resort to machine learning approaches. Some 2016). There are also some studies considering the methods use supervised approaches with hand-crafted whole image as input and performing a segmentation features or learned representations and some other in a single step as in (Brosch et al., 2015) and (Brosch methods use unsupervised approaches like clustering et al., 2016). In some methods global context is pro- which aim to detect lesion voxels as outliers. Ex- vided to the network, in addition to the local con- amples of supervised models used in MS segmenta- text, to give more information about the nature of a tion tasks are k-nearest neighbour methods, artificial voxel (Ghafoorian et al., 2017). Convolutional Neu- neural networks, random decision forests and baye- ral Networks (CNNs) are commonly used as part of sian frameworks among others (Garcia-Lorenzo et al., the architecture due to their strong feature extraction 2013). Examples of unsupervised models are fuzzy c- capabilities dealing with images (Vaidya et al., 2015), means or Gaussian mixture models with expectation while Restricted Boltzmann Machines (RBMs) and maximization (EM) (Garcia-Lorenzo et al., 2013). Auto-encoders are generally exploited to obtain a Unsupervised models suffer from non-uniformity in good initialization of the network, which might affect the image intensities and lesion intensities since this the ultimate performance, as shown in (Brosch et al., variability cannot be captured by a single global mo- 2015) and (Brosch et al., 2016). del (Havaei, 2016). In this respect supervised met- In this paper, we propose a MS lesion segmen- hods present an advantage, potentially being able to tation method based on a voxelwise classification on capture this variability with the appropriate choice of MRI with DL using a combination of different appro- training set or features. aches presented in the literature together with our own Recently, Deep Learning (DL) has been very contributions. We explore a new sub-sampling met- successful in the Computer Vision area, achieving hod to improve the learning process and develop a improvements in accuracies sometimes as high as new Convolutional Neural Networks (CNN) approach 30% (Plis et al., 2014). The main strength in DL, to achieve better performance results. The aim of this also differentiating it from other machine learning study is to achieve a method that will surpass the per- methods, is its automatic feature extraction capability. formance of existing methods in helping the experts Normally, raw data has to be processed automatically in the MS lesion segmentation work and even make or manually to extract meaningful and useful featu- their interruption minimal. We compare different ap- res through a process commonly known as ”feature proaches with DL so far applied to MS Segmentation. engineering”. This process requires time and careful analysis, and includes subjectivity, which might bias the results or produce erroneous results. However, 2 METHODOLOGY in DL, the feature extraction is data-driven using an appropriate loss function and learning algorithm for Deep Neural Networks, which removes the subjecti- In this section, we present our strategies for data vity, randomness and expert knowledge to a certain pre-processing, sub-sampling of the training set, de- degree. Moreover, the features obtained are hierarchi- signing the CNN architecture and developing diffe- cal, each network layer producing more abstract fea- rent approaches to improve the segmentation perfor- tures using the less abstract features obtained in the mance. previous layer. Thus feature extraction is carried out step-by- step, which is likelier to produce more com- 2.1 Data Pre-processing plex and useful features. Another strength of DL is its ability to represent very complex functions, which We have, at our disposal, T1 and T2 MRI modali- might also be considered as its drawback since it is ties, tissue segmentation and manual lesion segmenta- prone to easily over-fit. However, the over-fitting can tions of 59 subjects from Hospital Cl´ınic (Barcelona). be prevented with the correct guidance and regulariza- The tissue segmentation was performed by Freesurfer tion methods. DL methods are also robust to outliers, v.5.3.0 toolbox (Freesurfer, 2013). The manual seg- which is very common in neuroimaging data (Good- mentation was performed by an expert / two experts fellow et al., 2016), (Bengio, 2012) and (Deep Lear- from Hospital Cl´ınic team. The voxel resolution of ning, 2017). the MRIs is 0.86mm x 0.86mm x 0.86mm and the Previous work on MS lesion segmentation with image size is 208 x 256 x 256. As a pre-processing of DL is generally developed using voxelwise classifica- the MRI images we apply skull stripping, bias-field tion (lesion vs. normal) and is done on 2D/3D patches correction, tissue-segmentation and co-registration. centered on the voxel of interest to obtain a complete Additionally we apply 0-mean unit-variance normali- 261 VISAPP 2018 - International Conference on Computer Vision Theory and Applications zation to the data. For this process we use the Scikit- convolutional layer uses 60 kernels of size 4x4x4 with Learn’s preprocessing package (Scikit-Learn, 2010). a stride value of 1 and no padding. This layer is fol- lowed by an average pooling layer that takes 2x2x2 2.2 Sub-sampling Strategy patches of the output generated in the previous layer and produces an average for each patch. The stride We pose the segmentation problem as a voxelwise value for average pooling is 2.