(I123)FP-CIT reporting: Machine Learning, Effectiveness and Clinical Integration Jonathan Christopher Taylor Medical Physics Group Mathematical Modelling in Medicine Infection Immunity and Cardiovascular Disease Faculty of Medicine, Dentistry and Health The University of Sheffield Thesis submitted for the degree of PhD January 2019 Abstract (I123)FP-CIT imaging is used for differential diagnosis of clinically uncertain Parkinsonian Syndromes. Conventional reporting relies on visual interpretation of images and analysis of semi-quantification results. However, this form of reporting is associated with variable diagnostic accuracy results. The first half of this thesis clarifies whether machine learning classification algorithms, used as computer aided diagnosis (CADx) tool, can offer improved performance. Candidate machine learning classification algorithms were developed and compared to a range of semi-quantitative methods, which showed the superiority of machine learning tools in terms of binary classification performance. The best of the machine learning algorithms, based on 5 principal components and a linear Support Vector Machine classifier, was then integrated into clinical software for a reporting exercise (pilot and main study). Results demonstrated that the CADx software had a consistently high standalone accuracy. In general, CADx caused reporters to give more consistent decisions and resulted in improved diagnostic accuracy when viewing images with unfamiliar appearances. However, although these results were undoubtedly impressive, it was also clear that a number of additional, significant hurdles remained, that needed to be overcome before widespread clinical adoption could be achieved. Consequently, the second half of this thesis focuses on addressing one particular aspect of the remaining translation gap for (I123)FP-CIT classification software, namely heterogeneity of the clinical environment. Introduction of new technology, such as machine learning, may require new metrics, which in this work were informed through novel methods (such as the use of innovative phantoms) and strategies, enabling sensitivity testing to be developed, applied and evaluated. The pathway to acceptance of novel and progressive technology in the clinic is a tortuous one, and this thesis emphasises the importance of many factors in addition to the core technology that need to be addressed if such tools are ever to achieve clinical adoption. i Acknowledgements I would like to thank my supervisor, John Fenner, for always being supportive through the many challenges faced during this PhD. He was always willing to set aside time to look through my work or discuss the project, no matter how busy. I would also like to thank the National Institute for Health Research for providing me with the opportunity to undertake a PhD, and the Nuclear Medicine department at Sheffield Teaching Hospitals for allowing me to take time out from my clinical work. Finally, I would like to thank my partner Cat for her continual encouragement and support over the course of my fellowship, and for providing a welcome distraction from work when it was needed. ii Publications The work presented in this thesis contributed to a series of open-access, peer-reviewed publications (1–5). These are listed below along with the relevant chapters from which they originated. The license associated with each publication is also shown. 1) Taylor JC, Fenner JW. Comparison of machine learning and semi-quantification algorithms for (I123)FP-CIT classification: the beginning of the end for semi- quantification? EJNMMI Phys. 2017 Dec;4(1):29. (license CC-BY 4.0) CHAPTER 2 and 3 2) Taylor JC, Romanowski C, Lorenz E, Lo C, Bandmann O, Fenner J. Computer-aided diagnosis for (123I)FP-CIT imaging: impact on clinical reporting. EJNMMI Res. 2018 May 8;8(1):36. (license CC-BY 4.0) CHAPTER 4 3) Taylor JC, Vennart N, Negus I, Holmes R, Bandmann O, Lo C, et al. The subresolution DaTSCAN phantom: a cost-effective, flexible alternative to traditional phantom technology. Nucl Med Commun. 2018 Mar;39(3):268–75. (license CC-BY 4.0) CHAPTER 6 4) Taylor J, Fenner J. The challenge of clinical adoption—the insurmountable obstacle that will stop machine learning? BJR Open 2019; 1: 20180017. (license CC-BY 4.0) CHAPTER 5 and 8 5) Taylor JC, Fenner JW. Clinical Adoption of CAD: Exploration of the Barriers to Translation through an Example Application. Procedia Computer Science 2016 90:93-98. (license CC BY-NC-ND 4.0) CHAPTER 5 and 8 iii Statement of contribution The author declares that the work presented in this thesis is his own, with the exception of the following: The 3D printed phantom described in chapter 6, and the method for transforming the anatomical template to the dimensions of the phantom, was developed by colleagues at University Hospital Bristol. iv Glossary of terms SPECT Single Photon Emission Computed Tomography PS Parkinsonian Syndrome PD Parkinson's Disease ET Essential Tremor ((123)I-N-omega-fluoropropyl-2beta-carbomethoxy-3beta-(4- (123I)FP-CIT iodophenyl)nortropane PSP Progressive Supranuclear Palsy MSA Multiple system Atrophy DLB Dementia with Lewy Bodies CBD Corticobasal Degeneration VaP Vascular Parkinsonism DIP Drug Induced Parkinsonism AD Alzheimer's Disease HC Healthy Controls SWEDD Scans Without Evidence of Dopaminergic Deficit PDD Presynaptic Dopaminergic Deficit DaT Dopamine Active Transporters EANM European Association of Nuclear Medicine SNM Society of Nuclear Medicine SDDD Striatal Dopaminergic Deficit Disorder BNMS British Nuclear Medicine Society NHS National Health Service SBR Striatal Binding Ratio PPMI Parkinson's Progression Markers Initiative STH Sheffield Teaching Hospitals LEHR Low Energy High Resolution LEUHR Low Energy Ultra High Resolution PCA Principal Component Analysis LDA Linear Discriminant Analysis CNN Convolutional Neural Network ICA Independent Component Analysis SVM Support Vector Machine RBF Radial Basis Function v PNN Probabilistic Neural Network CV Cross Validation SVD Singular Value Decomposition SD Standard Deviation CI Confidence Interval DSC Dice Similarity Coefficient ShIRT Sheffield Image Registration Toolkit ROC Receiver Operator Curve ICC Intraclass correlation coefficient CAD(x) Computer Aided Diagnosis CADe Computer Aided Detection NICE National Institute for Health and Care Excellence IRAS Integrated Research Application System SSP Subresolution Sandwich Phantom FDM Fused Deposition Modelling PLA Polylactic Acid AAL Automated Anatomical Atlas MNI Montreal Neurological Institute ADNI Alzheimer‘s Disease Neuroimaging Initiative European Database of [123I]FP-CIT (DaTSCAN) SPECT scans of healthy ENCDAT controls MTEP Medical Technologies Evaluation Programme DAP Diagnostic Assessment Programme MICCAI Medical Image Computing and Computer Assisted Intervention MRI Magnetic Resonance Imaging FDA Food and Drug Administration vi Table of Contents Abstract.................................................................................................................................. i Acknowledgements ................................................................................................................ii Publications .......................................................................................................................... iii Statement of contribution ...................................................................................................... iv Glossary of terms .................................................................................................................. v Table of Contents ................................................................................................................. vii List of Figures ....................................................................................................................... xi List of Tables ....................................................................................................................... xv 1 Introduction .................................................................................................................... 1 1.1 (123I)FP-CIT imaging ............................................................................................. 1 1.1.1 Parkinsonian Syndromes ................................................................................. 1 1.1.2 Dementia with Lewy Bodies ............................................................................. 3 1.1.3 Tracer uptake and differential diagnosis ........................................................... 3 1.1.4 Clinical SPECT imaging ................................................................................... 6 1.1.5 Accuracy and variability of unaided visual analysis .......................................... 6 1.1.6 Conclusion ..................................................................................................... 10 1.2 Semi-quantification ............................................................................................... 11 1.2.1 Impact on clinical performance ...................................................................... 13 1.3 Machine Learning ................................................................................................. 16 1.3.1 Overview ........................................................................................................ 17 1.3.2 Automated classification for (I123)FP-CIT: a summary .................................. 18 1.4 Discussion and objectives ....................................................................................