F1000Research 2016, 5:1573 Last updated: 16 MAY 2019 STUDY PROTOCOL Automated analysis of retinal imaging using machine learning techniques for computer vision [version 1; peer review: 2 approved] Jeffrey De Fauw1, Pearse Keane1,2, Nenad Tomasev1, Daniel Visentin1, George van den Driessche1, Mike Johnson1, Cian O Hughes 1, Carlton Chu1, Joseph Ledsam1, Trevor Back1, Tunde Peto2, Geraint Rees3, Hugh Montgomery5, Rosalind Raine4, Olaf Ronneberger1, Julien Cornebise1 1DeepMind, London, EC4A 3TW, UK 2Moorfields Eye Hospital NHS Foundation Trust, London, EC1V 2PD, UK 3Alexandra House University College London, Bloomsbury Campus, London, WC1N 3AR, UK 4Department of Applied Heath Research, University College London, London, WC1E 7HB, UK 5Institute of Sport, Exercise and Health, London, W1T 7HA, UK First published: 05 Jul 2016, 5:1573 ( Open Peer Review v1 https://doi.org/10.12688/f1000research.8996.1) Latest published: 22 Jun 2017, 5:1573 ( https://doi.org/10.12688/f1000research.8996.2) Reviewer Status Abstract Invited Reviewers There are almost two million people in the United Kingdom living with sight 1 2 loss, including around 360,000 people who are registered as blind or partially sighted. Sight threatening diseases, such as diabetic retinopathy and age related macular degeneration have contributed to the 40% version 2 increase in outpatient attendances in the last decade but are amenable to published early detection and monitoring. With early and appropriate intervention, 22 Jun 2017 blindness may be prevented in many cases. version 1 Ophthalmic imaging provides a way to diagnose and objectively assess the published report report progression of a number of pathologies including neovascular (“wet”) 05 Jul 2016 age-related macular degeneration (wet AMD) and diabetic retinopathy. Two methods of imaging are commonly used: digital photographs of the fundus Yit Yang, Royal Wolverhampton NHS Trust, (the ‘back’ of the eye) and Optical Coherence Tomography (OCT, a 1 modality that uses light waves in a similar way to how ultrasound uses Wolverhampton, UK sound waves). Changes in population demographics and expectations and 2 Sandrine Zweifel, University Hospital Zürich, the changing pattern of chronic diseases creates a rising demand for such Zürich, Switzerland imaging. Meanwhile, interrogation of such images is time consuming, costly, and prone to human error. The application of novel analysis Any reports and responses or comments on the methods may provide a solution to these challenges. article can be found at the end of the article. This research will focus on applying novel machine learning algorithms to automatic analysis of both digital fundus photographs and OCT in Moorfields Eye Hospital NHS Foundation Trust patients. Through analysis of the images used in ophthalmology, along with relevant clinical and demographic information, Google DeepMind Health will Page 1 of 8 F1000Research 2016, 5:1573 Last updated: 16 MAY 2019 clinical and demographic information, Google DeepMind Health will investigate the feasibility of automated grading of digital fundus photographs and OCT and provide novel quantitative measures for specific disease features and for monitoring the therapeutic success. Keywords Optical Coherence Tomography, machine learning, artificial intelligence, diabetic retinopathy, neovascular age-related macular degeneration, ophthalmology, retina This article is included in the Machine learning: life sciences collection. Corresponding author: Joseph Ledsam ([email protected]) Competing interests: Moorfields Eye Hospital NHS Foundation Trust administration time spent on this work will be paid to the trust. The Chief Investigator and some co-investigators are paid employees of Google DeepMind. Several co-investigators (PK, JL, GR, RR) are paid contractors for Google DeepMind. Grant information: Google DeepMind is funding the research. Copyright: © 2016 De Fauw J et al. This is an open access article distributed under the terms of the Creative Commons Attribution Licence, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited. How to cite this article: De Fauw J, Keane P, Tomasev N et al. Automated analysis of retinal imaging using machine learning techniques for computer vision [version 1; peer review: 2 approved] F1000Research 2016, 5:1573 (https://doi.org/10.12688/f1000research.8996.1) First published: 05 Jul 2016, 5:1573 (https://doi.org/10.12688/f1000research.8996.1) Page 2 of 8 F1000Research 2016, 5:1573 Last updated: 16 MAY 2019 Background eye health services in the UK and a world-class centre of excellence Age-related macular degeneration (AMD) is a degenerative reti- for ophthalmic research and education. Should the research be suc- nal disease that can cause irreversible visual loss (Bressler, 2004). cessful, implementation of the outcomes would improve patient It is the leading cause of blindness in Europe and North America access to treatment and ease pressures on time and resources in and accounts for over half of partially sighted or legally blind cer- ophthalmology clinics. tifications in the UK (Bunce, 2010). Neovascular (“wet”) AMD is an advanced form of macular degeneration that historically has Aims and objectives accounted for the majority of vision loss related to AMD. It is Primary objective characterised by abnormal blood vessel growth that can result in 1.1 hemorrhage, fluid exudation and fibrosis, and thus to local macular Exploratory study: investigate whether computer algorithms can damage and ultimately vision loss (Owen, 2012). detect and classify pathological features on eye imaging, including fundus digital photographs and OCT. Diabetic retinopathy (DR) is the leading cause of blindness in working age populations in the developed world (Cheung, 2010). Secondary objectives It is estimated that up to 50% of people with proliferative DR If the exploratory study is successful: (characterised by neovascularisation) who do not receive timely treatment will become legally blind within 5 years (Shaw, 2010). 2.1 Although up to 98% of severe visual loss due to DR can be pre- To provide novel image analysis algorithms to identify and quantify vented with early detection and treatment, once it has progressed specific pathological features in eye imaging, using validated meth- vision loss is often permanent (Kollias, 2010). Indeed, 4,200 ods and expert clinical consensus. people in England every year are at risk of blindness caused by diabetic retinopathy and there are 1,280 new cases of blindness 2.2 caused by diabetic retinopathy (Scanlon, 2008). To provide quantitative measurements disease progression, severity and to monitor the therapeutic success over time. To diagnose these conditions and monitor their progression (and response to treatment) the presence and precise location of the Study design lesions must be determined. Two imaging modalities are com- This is a retrospective, non-interventional exploratory study. monly used for this purpose: digital photographs of the fundus (the Analyses performed in the study will be on fully anonymised (to ‘back’ of the eye) and Optical Coherence Tomography (OCT, a achieve the primary objective) and pseudonymised (to achieve the modality that uses light waves in a similar way to how ultrasound secondary objective 2.2) retinal images (including fundus images uses sound waves) (Huang, 1991). and OCT). These images will contain no patient identifiable information. Clinics such as these have generated very large datasets of both digital fundus and OCT images. They are also very busy; oph- Inclusion criteria thalmology clinics often have long waits and average times to All patients attending Moorfields Eye Hospital NHS Foundation urgent first treatments can be greater than two weeks in the busiest Trust sites between 01/01/2007 and 29/02/2016, and who had clinics. digital retinal imaging (including fundus digital photographs and OCT) as part of their routine clinical care, will be eligible for inclu- Machine learning algorithms make use of the rich, varied datasets sion in this study. to find high dimensional interactions between multiple data points (Murphy, 2012). Most machine learning methods can be thought Exclusion criteria of as a form of statistics. Some algorithms look for patterns in Hard copy examinations (i.e. physical photograph copies prior to the data, identifying subgroups within the full sample (known as digital storage) will be ineligible, and will be excluded by the nature clustering). Others rely on specific clinical information against of the original service evaluation requests from the Moorfields which the algorithm compares its predictions and adjusts accord- team. Data from patients who have previously manually requested ingly (supervised learning). that their data should not be shared, even for research purposes in anonymised form, and have informed Moorfields Eye Hospital of There have been significant recent advances in the field of machine this, will be ineligible and removed by Moorfields Eye Hospital learning demonstrating algorithms able to learn how to accom- staff before research begins. plish tasks without instruction (Mnih, 2015; Silver, 2016). Recent healthcare applications of such algorithms have shed light on Sample size complex genetic interactions in autism (Uddin, 2014) and monitor- Approximately 1 million examinations meet the above criteria. ing of physiological observations in intensive care (Clifton, 2013). Most recent machine learning algorithms benefit