Artificial Intelligence in Artificial Intelligence in Cancer Care Cancer Care Artificial Intelligence? J. Ross Mitchell, PhD, MSc Artificial Intelligence Officer, Moffitt Cancer Center Machine Learning? Deep Learning? AAMD Annual Meeting, July, 2020 1 2 AI in Cancer Care - July 5, 2020 Artificial Intelligence (1956): • Mimic human cognition • Rule-based systems Why Now? Machine Learning (1980s): • Algorithms learn by example (from data) 1. Computing power 2. Data 3. Learning Algorithms Deep Learning (2000s): • Algorithms use deep networks of artificial neurons to learn from data 4 3 4 AI in Cancer Care - July 5, 2020 1: Computing Power: 1993 - 2020 7M inches = 110.5 miles 100,000,000 10,000,000 1,000,000 100,000 7M x #1 Supercomputer Worldwide 10,000 Performance (GFLPOS) 1,000 107 miles 100 10 Feb 7, 2000 Jun 2, 2020 May 1, 1993Aug 3, 1995Nov 5, 1997 May 12, 2002Aug 13, 2004Nov 16, 2006Feb 17, 2009May 23, 2011Aug 24, 2013Nov 27, 2015Feb 28, 2018 5 Source: top500.org 6 5 6 AI in Cancer Care - July 5, 2020 1: Computing Power: 1993 - 2019 #1 Supercomputer: 1997 100,000,000 10,000,000 1,000,000 iPhone 11 100,000 7M x #1 Supercomputer Worldwide 10,000 Performance (GFLPOS) 1,000 100 ASCI Red: • Sandia National Lab 10 • 6,000 Pentium Pros @ 200 MHz • Size: tennis court Feb 7, 2000 Jun 2, 2020 May 1, 1993Aug 3, 1995Nov 5, 1997 May 12, 2002Aug 13, 2004Nov 16, 2006Feb 17, 2009May 23, 2011Aug 24, 2013Nov 27, 2015Feb 28, 2018 • Energy: 800 homes • Cost: $50 M Source: top500.org 7 Source: top500.org 7 8 AI in Cancer Care - July 5, 2020 2: Data 2: The Unreasonable Effectiveness of Data “Hospitals have enormous amounts of data, which is inaccessible to anyone except for themselves. These machine learning techniques allow you to take all of that information, sum it all together,should and actuallyallow produce outcomes.” - Eric E. Schmidt, PhD. Former Executive Chairman and Google CEO. Speaking @ MIT, May 3, 2017 Transactions of the Association for Computational Linguistics, 2001 http://news.mit.edu/2017/eric-schmidt-visits-mit-to-discuss-computing-ai-future-of-technology-0504 9 9 10 AI in Cancer Care - July 5, 2020 2: The Unreasonable Effectiveness of Data 2: The Unreasonable Effectiveness of Data “Complex / Deep” “ComplexJuly 11, 2016 / Deep” “Simple / Handcrafted”1. When N large, choice of “Simple / Handcrafted” algorithm has small impact Accuracy 2. When N small, simple may Accuracy be better than complex In other words… 3. Accuracy improves 3. WhenAccuracy N small, improves invest your effort asymptotically inasymptotically collecting & labeling more data! Millions of Words Millions of Words 11 12 AI in Cancer Care - July 5, 2020 3: Algorithms 3: Algorithms Universal Approximation Theorem (1989) 200 neurons x y ‘We ceased to be the lunatic fringe. We’re now the lunatic core.’ Nature Vol. 323 9 October 1986 — Geoff Hinton, U. of Toronto © 1986 Nature Publishing Group 13 Source: http://neuralnetworksanddeeplearning.com/chap4.html 14 Source: https://www.wired.com/2014/01/geoffrey-hinton-deep-learning/ 13 14 AI in Cancer Care - July 5, 2020 Putting it all Together… Image Classification Accuracy Over Time Four Cancer Applications & 100% Deep Learning used 96.4% Predictions… 95% Human Expert 93.3% 90% 88.3% Typical Human 85% 83.6% Accuracy "We always overestimate the change that 80% will occur in the next two years and underestimate the change that will occur 75% 74.2% in the next ten." 71.8% 70.5% - Bill Gates, “The Road Ahead” 1996. 70% 2009 2010 2011 2012 2013 2014 2015 Year 14M Images. Crowdsourced labels. 22k Categories (plant, animal, device, food, structure, person) 15 16 15 16 AI in Cancer Care - July 5, 2020 Radiomics Application #1: Normal Abnormal Radiomics Feature Extractor Normal Features Abnormal Features Regression Normal Abnormal 18 17 18 AI in Cancer Care - July 5, 2020 Radiomics History: Cats vs Dogs (2007) Prove you are human: Unknown Identify cats from 12 photos of cats and dogs* “User studies indicate it can be solved by humans 99.6% of the time in under 30 seconds.” Feature Extractor “We believe [computer] classification accuracy of better than 60% will be Unknown difficult without a significant advance in the state of the art.” Regression Normal Abnormal * J. Elson, J. Douceur, J. Howell and J. Saul. Asirra: a CAPTCHA that exploits interest-aligned manual image categorization. In Proc. of ACM CCS 2007, pp. 366–374. 19 20 19 20 AI in Cancer Care - July 5, 2020 Kaggle Dogs vs Cats (2013) Visualizing CNN Layers VGG 16 2014: 99% Accuracy https://www.kaggle.com/c/dogs-vs-cats/overview 21 22 AI in Cancer Care - July 5, 2020 VGG 16 (Layer 7) VGG 16 (Layer 14) 23 https://towardsdatascience.com/how-to-visualize-convolutional-features-in-40-lines-of-code-70b7d87b0030 24 https://towardsdatascience.com/how-to-visualize-convolutional-features-in-40-lines-of-code-70b7d87b0030 23 24 AI in Cancer Care - July 5, 2020 VGG 16 (Layer 30) VGG 16 (Layer 40) 25 https://towardsdatascience.com/how-to-visualize-convolutional-features-in-40-lines-of-code-70b7d87b0030 26 https://towardsdatascience.com/how-to-visualize-convolutional-features-in-40-lines-of-code-70b7d87b0030 25 26 AI in Cancer Care - July 5, 2020 Training Cats vs Dogs (2020) Training Cats vs Dogs (2020) No Code! Create ML Create ML 27 28 27 28 AI in Cancer Care - July 5, 2020 Predictions… 2 - 10 Years Application #2: Normal Abnormal Contouring Feature Extractor Deep Radiomics: Normal Features Abnormal Features • Learned Deep Neural Network • Automatic • High-level 1,000,000’s • Transferable Regression Normal Abnormal 29 29 30 AI in Cancer Care - July 5, 2020 Deep Learning for (Brain) Tumor Contouring: Brain Tumors (3D CNN) Segmentation • Moffitt Cancer Center, Tampa USA • Imperial College, London UK • Mayo Clinic, Phoenix USA ross.mitchell@moffitt.org Overlap with Technician Segmentations: 89% 32 31 31 32 AI in Cancer Care - July 5, 2020 Contouring Time Results (elapsed time: 1 - 2 minutes) Processing Pipeline* (Python 3.6): Time (s) 1. Image processing 36 2. Brain segmentation (CNN) 14 3. Label brain regions 35 4. Tumor segmentation (CNN) 8 Total Time 93 Total Time (no labeling of brain regions) 58 *Running on Amazon Web Services p3.2xlarge Instance(s) 33 34 AI in Cancer Care - July 5, 2020 Contouring: Brain Tumors (3D CNN) Contouring: Brain Tumors (3D CNN) Tumor Type N % Glioblastomas 463 62.5% 1 Glioblastoma Multiforme 449 2 Glioblastoma Multiforme with Oligodendroglial Component 7 Study Source N Age M / F (Not Specified) Study Dates 3 Giant Cell Glioblastoma 4 4 Glioblastoma Multiforme, Small Cell Type 2 1 Cancer Centers (n=8) 525 53.1 ± 15.9 338 / 187 2000 - 2016 (2008) 5 Glioblastoma Multiforme with Sarcomatous Differentiation 1 Astrocytomas 77 10.4% 2 TCGA-GBM 101 58.4 ± 14.4 63 / 38 1996 - 2008 (2001) 6 Astrocytoma 38 7 Anaplastic Astrocytoma 28 3 TCIA 85 45.6 ± 15.6 33 / 24 (28) 1990 - 2005 (1994) 8 Diffuse Astrocytoma 7 9 Infiltrating Fibrillary Astrocytoma 2 4 Ivy GAP* 18 56.7 ± 13.4 7 / 11 1996 - 2000 (1997) 10 Gemistocytic Astrocytoma 1 11 Pleomorphic Xanthoastrocytoma 1 Oligodendrogliomas 37 5.0% 5 Radiation Therapy Oncology 12 66.9 ± 17.0 10 / 2 2009 - 2011 (2010) 12 Oligodendroglioma 27 Group 13 Anaplastic OligodendroGlioma 10 Overall 741 53.5 ± 16.0 451 / 262 (28) 1990 - 2016 (2006) Mixed and Other 19 2.5% 14 Anaplastic OligoAstrocytoma 9 Table 1: Primary sources for the exams processed in this study. In total, 8 different North American academic cancer centers, 2 public domain datasets, and 2 foundation sponsored studies contributed exams. ‘Study Source’ indicates the origin of the MRI exams. ‘N’ 15 Gliosarcoma 5 indicates the number of exams contributed. ‘Age’ is the mean age (± standard deviation) of the patients when the exam was obtained. ‘M / 16 OligoAstrocytoma 2 F (Not Specified)’ indicates the number of male (M) and female (F) patients in the group. The number of patients whose sex was not specified is indicated in brackets. ‘Study Dates’ lists the range of years the exams were acquired, with the median year indicated in 17 GanglioGlioma 1 brackets. The last row provides summary values for the entire cohort. * Ivy Glioblastoma Atlas Project 18 Diffuse Pontine Intrinsic Glioma 1 19 Low Grade Glioma 1 Not Specified 145 19.6% Total 741 100% 35 36 AI in Cancer Care - July 5, 2020 Web-portal for neurorads to adjudicate results! Contouring Quality 20 Neuro-rads Performed 400 Adjudications Mean Median Score* Score Technician 6.97 7 Proportion DeepMedic 7.31 8 Difference: 0.34 *p < 0.00007 Score • Deep Learning Model is better than the technicians • But: the technicians created the training data • How can a model be better than the data used to train it? 37 38 AI in Cancer Care - July 5, 2020 Models & Training Data Label Quality Label Accuracy Model Accuracy The Essential Guide to Training Data Why Your AI Model is Only as Good as Its Training Data Number of Training Samples KDD '08: Proceedings of the 14th ACM SIGKDD international conference on Knowledge discovery and data mining; August 2008; Pages 614–622; https://doi.org/10.1145/1401890.1401965 39 40 An Appen eBook AI in Cancer Care - July 5, 2020 Predictions… Application #3: 2 - 10 Years •Imperfect (noisy) labels are OK (accuracy > 75%) Therapy •Models can learn to see through the noise with few samples •Automatic contouring of all lesions & organs •Better than technicians; 30 seconds or less •Human input limited to review & editing of exceptions •Human-in-the-loop (adaptive) learning 41 41 42 AI in Cancer Care - July 5, 2020 Reinforcement Learning Reinforcement Learning “Reinforcement learning is learning from rewards, by trial and error, during normal interaction with the world.