Deep Ensemble Learning Based Objective Grading of Macular Edema by Extracting Clinically Significant Findings from Fused Retinal
Total Page:16
File Type:pdf, Size:1020Kb
sensors Article Deep Ensemble Learning Based Objective Grading of Macular Edema by Extracting Clinically Significant Findings from Fused Retinal Imaging Modalities 1, 2, 3, 4 5 Bilal Hassan y , Taimur Hassan y , Bo Li *, Ramsha Ahmed and Omar Hassan 1 School of Automation Science and Electrical Engineering, Beihang University (BUAA), Beijing 100191, China 2 Department of Electrical Engineering, Bahria University (BU), Islamabad 44000, Pakistan 3 School of Computer Science and Engineering, Beihang University (BUAA), Beijing 100191, China 4 School of Computer and Communication Engineering, University of Science & Technology Beijing (USTB), Beijing 100083, China 5 Department of Electrical and Computer Engineering, Sir Syed CASE Institute of Technology (SSCIT), Islamabad 44000, Pakistan * Correspondence: [email protected] These authors contributed equally in this research. y Received: 26 May 2019; Accepted: 26 June 2019; Published: 5 July 2019 Abstract: Macular edema (ME) is a retinal condition in which central vision of a patient is affected. ME leads to accumulation of fluid in the surrounding macular region resulting in a swollen macula. Optical coherence tomography (OCT) and the fundus photography are the two widely used retinal examination techniques that can effectively detect ME. Many researchers have utilized retinal fundus and OCT imaging for detecting ME. However, to the best of our knowledge, no work is found in the literature that fuses the findings from both retinal imaging modalities for the effective and more reliable diagnosis of ME. In this paper, we proposed an automated framework for the classification of ME and healthy eyes using retinal fundus and OCT scans. The proposed framework is based on deep ensemble learning where the input fundus and OCT scans are recognized through the deep convolutional neural network (CNN) and are processed accordingly. The processed scans are further passed to the second layer of the deep CNN model, which extracts the required feature descriptors from both images. The extracted descriptors are then concatenated together and are passed to the supervised hybrid classifier made through the ensemble of the artificial neural networks, support vector machines and naïve Bayes. The proposed framework has been trained on 73,791 retinal scans and is validated on 5100 scans of publicly available Zhang dataset and Rabbani dataset. The proposed framework achieved the accuracy of 94.33% for diagnosing ME and healthy subjects and achieved the mean dice coefficient of 0.9019 0.04 for accurately extracting the retinal fluids, 0.7069 0.11 for ± ± accurately extracting hard exudates and 0.8203 0.03 for accurately extracting retinal blood vessels ± against the clinical markings. Keywords: biomedical image processing; image analysis; image classification; machine intelligence; machine vision; optical coherence tomography; fundus photography 1. Introduction Visual impairments severely degrade the quality of life and have an adverse effect on people suffering from other chronic health issues. Currently, blindness is considered as a major health problem worldwide. According to the Global Burden of Disease (GBD) in their 2017 report (released on 18 November 2018), loss of vision is categorized as the third leading form of impairments in humans and 48.2 million people are suffering from eye diseases all over the world. In addition to Sensors 2019, 19, 2970; doi:10.3390/s19132970 www.mdpi.com/journal/sensors Sensors 2019, 19, x FOR PEER REVIEW 2 of 25 Visual impairments severely degrade the quality of life and have an adverse effect on people suffering from other chronic health issues. Currently, blindness is considered as a major health problem worldwide. According to the Global Burden of Disease (GBD) in their 2017 report (released on November 8th, 2018), loss of vision is categorized as the third leading form of impairments in humans and 48.2 million people are suffering from eye diseases all over the world. In addition to this, 39.6 million people have severe visual impairments whereas 279 million people and 969 million people have moderate to low visual impairments, respectively [1,2]. Moreover, most of the visual impairments that were reported are due to retinopathy. The prime cause of retinopathy is diabetes mellitus (DM). DM is caused due to the destruction of pancreaticSensors beta2019, 19 cells, 2970 (β-cell) affecting the glucose metabolism of the candidate subject.2 of 26 DM is graded into two types. Type I DM specifies deficiency of insulin whereas Type II DM is associated with insulinthis, resistance 39.6 million [3–5]. people haveApart severe from visual this, impairments DM also whereasaffects 279other million vital people organs and of 969 the million human body people have moderate to low visual impairments, respectively [1,2]. Moreover, most of the visual including eyes, kidney, heart, etc. [6]. Macula produces the central vision and it is the most critical impairments that were reported are due to retinopathy. part of retina.The Any prime damage cause of to retinopathy macula results is diabetes in mellitus the loss (DM). of central DM is caused vision. due The to the retinal destruction diseases that affect the ofcentral pancreatic vision beta of cells a person (β-cell) are affecting collect theively glucose known metabolism as maculopathy. of the candidate The subject. most DM common is form of maculopathygraded intois ME, two types.which Type is caused I DM specifies by the deficiency leakage of of insulin extracellular whereas Type fluid II DM from is associated hyper-permeable with insulin resistance [3–5]. Apart from this, DM also affects other vital organs of the human body capillaries in the macula of the retina. ME is clinically graded into different stages depending upon including eyes, kidney, heart, etc. [6]. Macula produces the central vision and it is the most critical the affectedpart area of retina. of macular Any damage thickening. to macula resultsHowever, in the early loss of centraldetection vision. and The laser retinal photocoagulation diseases that can prevent suddenaffect the blindness central vision in of most a person of are the collectively cases. Moreover, known as maculopathy. many retinal The most complications common form are often treatable andof maculopathy according is to ME, the which initiative is caused of by“VISIO the leakageN 2020: of extracellular The Right fluid to Sight”, from hyper-permeable different measures are capillaries in the macula of the retina. ME is clinically graded into different stages depending upon the being taken to eradicate avoidable blindness by the year 2020 [7]. At the same time, it is equally affected area of macular thickening. However, early detection and laser photocoagulation can prevent importantsudden to equip blindness ophthalmologists in most of the cases. with Moreover, state-of-t manyhe-art retinal retinal complications computer are oftenaided treatable diagnostic and systems for efficientaccording detection to the and initiative grading of “VISION of retinopathy. 2020: The Right to Sight”, different measures are being taken to The twoeradicate non-invasive avoidable blindness imaging by modalities the year 2020 [that7]. At are the clinically same time, itin is practice equally important for retinal to equip examination, are OCT andophthalmologists fundus imagery with state-of-the-art [8]. OCT retinalcaptures computer the aidedtissue diagnostic reflection systems through for efficient light detection coherency. For and grading of retinopathy. retinal examination,The two a non-invasive beam is bombarded imaging modalities on the that fundus are clinically of retina in practice yielding for retinal a cross-sectional examination, are axial scan (A-scan) [9–10].OCT and The fundus A-scans imagery are [8]. joined OCT captures together the tissue to produce reflection througha brightness light coherency. scan (B-scan). For retinal Since OCT captures theexamination, cross-sectional a beam isretina, bombarded so early on the progre fundusssion of retina of retinopathy yielding a cross-sectional can be easily axial visualized. scan The early identification(A-scan) [9,10 of]. Theretinopathy A-scans are joinedeffectively together leads to produce towards a brightness the better scan (B-scan). treatment. Since OCTRetinal OCT captures the cross-sectional retina, so early progression of retinopathy can be easily visualized. The early imagery hasidentification revolutionized of retinopathy the clinical effectively examination leads towards and the bettereye treatment treatment. Retinal [11,12]. OCT Figure imagery 1 (a) has shows the basic OCTrevolutionized scan acquisition the clinical schematics, examination which and eye is treatment based [11on,12 Michelson]. Figure1a shows interferometer the basic OCT scan(MI). In MI, a monochromaticacquisition coherent schematics, light which source is based is used on Michelson to penetrate interferometer the human (MI). eye In MI,to produce a monochromatic a cross sectional retinal scan.coherent Beam light splitter source at is usedthe tocenter penetrate splits the the human light eye source to produce into a crosstwo sectionalseparate retinal beams scan. where one Beam splitter at the center splits the light source into two separate beams where one beam is directed beam is directedtowards a towards reference mirror a reference and the other mirror travels and to thethe subject’s other eye.travels These to two the beams subject’s upon reflection eye. These two beams uponget recombinedreflection intoget arecombined