Bhandari et al. Insights into Imaging (2020) 11:77 https://doi.org/10.1186/s13244-020-00869-4 Insights into Imaging EDUCATIONAL REVIEW Open Access Convolutional neural networks for brain tumour segmentation Abhishta Bhandari1,2*, Jarrad Koppen1 and Marc Agzarian3,4 Abstract The introduction of quantitative image analysis has given rise to fields such as radiomics which have been used to predict clinical sequelae. One growing area of interest for analysis is brain tumours, in particular glioblastoma multiforme (GBM). Tumour segmentation is an important step in the pipeline in the analysis of this pathology. Manual segmentation is often inconsistent as it varies between observers. Automated segmentation has been proposed to combat this issue. Methodologies such as convolutional neural networks (CNNs) which are machine learning pipelines modelled on the biological process of neurons (called nodes) and synapses (connections) have been of interest in the literature. We investigate the role of CNNs to segment brain tumours by firstly taking an educational look at CNNs and perform a literature search to determine an example pipeline for segmentation. We then investigate the future use of CNNs by exploring a novel field—radiomics. This examines quantitative features of brain tumours such as shape, texture, and signal intensity to predict clinical outcomes such as survival and response to therapy. Keywords: Glioblastoma, Convolutional neural network, Artificial intelligence, Segmentation Keypoints new frontier for radiology. It is important for radiolo- gists to be abreast of advances in machine learning. Convolutional neural networks simply involve This has been recognised by the recent changes in analysing features derived from the image to the Royal Australian and New Zealand College of Ra- perform tasks such as segmenting tumours. diologists (RANZCR) curriculum that incorporates This initially involves training the network with a machine learning into the part I applied imaging manually segmented dataset which then is poised to technology examinations [1]. Methods that incorpor- segment patient images. atequantitativeanalyseswilladdtothetraditional This has a role in segmentation of brain tumours visual analysis of images. An important step in the such as glioblastoma and lower-grade astrocytomas. image analysis pipeline is the anatomical segmentation Segmented images can be further processed to of regions of interest (ROI), for example, defining a predict clinical sequelae such as survival and volume of abnormal tissue from a background of nor- response to therapy. mal tissue. This will allow for statistical analysis of features that is not visible by human perception [2]. Introduction For example, the field of radiomics is fast developing With the introduction of methods to quantitatively as a method of predicting survival times from imaging analyse gliomas with computational methods comes a features such as shape of a volume of interest and texture and intensity of the voxel habitat. With the * Correspondence: [email protected] development of these methods comes a greater need 1Townsville University Hospital, Townsville, Queensland, Australia for automated segmentation. Figure 1 shows incon- 2Department of Anatomy, James Cook University, Townsville, Queensland, Australia sistencies in blinded manual segmentation of brain Full list of author information is available at the end of the article tumours by the first and second authors. A measure © The Author(s). 2020 Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/. Bhandari et al. Insights into Imaging (2020) 11:77 Page 2 of 9 produce the expected result [7]. Further details will be provided in this paper. From Fig. 2, each blue circle represents a node or neuron from which the name ‘neural network’ is derived from. There is an input to each neuron. The arrows or ‘axons’ represent the connection between neurons. The result is an output which generates an approximation of the image which is iteratively refined [8]. The nodes receive an initial input from a data source—as seen in Fig. 2. This is then fed into the next neuronal layer and given an initial weighting. This mid- dle ‘hidden’ layer can be repeated a multitude of times. This is then fed into the output node, and this produced the desired result. However, this needs to be refined fur- ther, and one loop or iteration is not sufficient for the generation of an optimal output. This is where the nov- elty of a neural network comes in. In order to refine the output of the nodes, the weightings are changed. Thus, through iteration, the nodes are given different weight- Fig. 1 Manual anatomical segmentation by first (red) and second ings. Based on these weightings, the output can changed author (yellow) with intersection (purple). Image courtesy of A.Prof through each iteration and eventually an output that Frank Gaillard, Radiopaedia.org, rID: 22205 reaches the desired result can be produced. Thus, neural networks provide a means of optimising an initial data of consistency of image segmentation can be per- input via weighting certain aspects of the input to pro- formed by the Sørensen–Dice coefficient, and this duce an optimal result. was calculated with the StudierFenster calculator There are various other segmentation methods de- (available at: http://studierfenster.tugraz.at/). This tailed elsewhere [9]. Some of the notable segmentation ranges from 0 to 1 with 1 having 100% consistency models are: [3]. The value obtained from the segmentation by the first and second author was 0.91 which demonstrates Thresholding method—as the name implies, voxels the discrepancy in manual segmentation. above a threshold are classified as belonging to the As an example of machine learning, this educational tumour [10]. paper will examine the use of convolutional neural net- Edge-based method—changes in the intensity works for low-grade diffuse astrocytoma (World Health between edges of voxels are used as the boundaries Organization grade 2) and high-grade (World Health of the tumours [11]. Organization grade 4) glioblastoma—also known as glio- Region growing method—a seed voxel is inputted blastoma multiforme (GBM) segmentation. Convolutional into the segmentation; from this seed, voxels that neural networks (CNNs) are a unique machine learning are similar are classified as belonging to the structure originally modelled on the human visual cortex tumour [12]. [4]. The brain was studied due to the abundance of seg- Watershed algorithm—this is a unique segmentation mentation methods that are already available and well method whereby the voxel intensities or gradients established in the literature [5]. Machine learning is fast are represented by a topographical map similar to developing and is exponentially being represented at inter- those seen in geography. Based on the ‘steepness’ of national conferences [6]. An educational perspective is the map, a boundary is assigned [13]. needed for radiologists. This paper provides a novel bal- Atlas method—a tumour free reference MRI is used ance between education and a state-of-the-art review on to segment the MRI containing the tumour volume convolutional neural networks in glioblastoma. [14]. To better understand CNNs, artificial neural net- works will be reviewed briefly as this is a simple The advantages of the convolutional neural network are introduction for understanding neural networks such the fact that it provides optimal accuracy of segmentation. as CNNs. Artificial neural networks involve inputs However, this is at the cost of computational load [9]. With which feed into a hidden layer which has biases or advances in computation, the implementation of convolu- weightings associated with it and outputs which tional neural networks and refinement of the structural seg- change as the machine ‘learns’ from a dataset to mentation of brain tumours can be enhanced. Bhandari et al. Insights into Imaging (2020) 11:77 Page 3 of 9 Table 1 CNN studies and main findings obtained from literature search Author, year Title Main findings Pereira, 2016 Brain Tumour Segmentation Using Convolutional Neural Small 3 × 3 kernels for convolution to combat overfitting. DSC— [30] Networks in MRI Images complete tumour, 0.78; core tumour, 0.65; and enhancing regions, 0.75 Arunachalam, An efficient and automatic glioblastoma brain tumor detection Unique segmentation process involving SIST and NSCT 2017 [23] using shift-invariant shearlet transform and neural networks transformation to convert the image into a multi-resolution image. Standard feature extraction occurs. Accuracy is reported at 99.8% for the proposed method Havaei, 2017 Brain tumour segmentation with Deep Neural Networks The TwoPathCNN (focusing