Radiology in the Era of Artificial Intelligence November 11, 2019 Curtis P. Langlotz, MD, PhD Professor of Radiology and Biomedical Informatics, Stanford University Director, Center for Artificial Intelligence in Medicine & Imaging Associate Chair, Information Systems, Department of Radiology Medical Informatics Director, Stanford Health Care @curtlanglotz Disclosures
• Shareholder and advisor: • whiterabbit.ai • Nines.ai • GalileoCDS, Inc • Bunker Hill, Inc • Department research support: • Philips • Siemens Healthineers • GE Healthcare • School of Medicine research support: • Google • Board of directors, Radiological Society of North America Variability in Response Assessment
Schwartz LH et al. Variability in Response Assessment in Solid McNitt-Gray MF et al. Computed Tomography Assessment of Response to Therapy: Tumors: Effect of Number of Lesions Chosen for Measurement. Tumor Volume Change Measurement, Truth Data, and Error. Transl Oncol. 2009 Dec 1; Clin Cancer Res. 2003 Oct 1;9(12):4318-23. 2(4): 216–222. http://www.image-net.org/
• 14 million images
• 21,841 distinct labels:
• 856 types of bird
• 993 types of tree
• 157 musical instruments
• Russakovsky O, Deng J, Su H, et al. ImageNet Large Scale Visual Recognition Challenge. Int J Comput Vis. 2015;115(3):211-252. • https://www.economist.com/news/special-report/21700756-artificial-intelligence-boom-based-old-idea-modern-twist-not • http://karpathy.github.io/2014/09/02/what-i-learned-from-competing-against-a-convnet-on-imagenet/ Dramatic Advances in Accuracy of Deep Learning
ImageNet Visual Recognition Error Rates 30%
25%
20%
15%
10% Human error rate 5%
0% 2010 2011 2012 2013 2014 2015 2016 2017
Russakovsky O, Deng J, Su H, Krause J, Satheesh S, Ma S, et al. ImageNet Large Scale Visual 5 Recognition Challenge. Int J Comput Vis. 2015;115: 211–252. http://www.radiologyassistant.nl/ Karpathy, Andrej & Li, Fei Fei. Deep Visual-Semantic Alignments for Generating Image Descriptions, CVPR, 2015 Advances in Computation
ASR 33 Teletype, 1976 DEC PDP-8, 1965 Apple A12 Processor, 2019 75 lbs, 110 baud 0.333 MHz, $18,500 2.49 GHz x 6, $1000
https://www.pdp8.net/asr33/asr33.shtml Advances in Storage
IBM 5MB HD, 1956 Seagate 4TB HD, 2019 $3200/month lease $99, Amazon.com Neural Network A flexible and powerful form of machine learning.
Positive Cases Malignant
Negative Cases
Benign
https://hackernoon.com/challenges-in-deep-learning-57bbf6e73bb “Deep” Neural Networks: Tens of Millions of Parameters Learning Object Recognition
Faces Cars Elephants Chairs
Courtesy of Andrew Ng Machine Learning Research in Medical Imaging
Source “Raw” Data New Labeled Training Data Decision Image Support Actionable Recon Systems New New Advice Methods Machine Machine Learning Learning Explanation Methods Methods New Image Labeling Methods
CT scan icon by Sergey Demushkin from the Noun Project • Federated learning • Generative adversarial networks • Semi-supervised learning • Deep contextualized word representations • Multi-task learning • Reinforcement learning • Explainable AI models • Homomorphic encryption Foundational AI Research in Radiology: Ingredients for Success
Healthcare Academic AI Established Delivery Professional Health Ingredient for Success Startups Companies Systems Societies Systems Deep technical knowledge High performance computing Interdisciplinary teams Ongoing source of labeled images Infrastructure for prospective evaluation Market dissemination channel
=available =can acquire =difficult to acquire • Anesthesia • Bioengineering • Biomedical Data Science • Cardiothoracic Surgery • Computer Science • Dermatology • Emergency Medicine • Genetics • Medicine • Neurology & Neurological Sciences • Neurosurgery • Ophthalmology • Pathology • Pediatrics • Psychiatry & Behavioral Sciences • Psychology • Radiation Oncology • Radiology • Surgery • Urology
Key Computer Science Faculty Collaborators
Fei Fei Li, PhD Chris Manning, PhD Andrew Ng, PhD Creator of ImageNet Stanford AI Lab Director Deep Learning Pioneer AIMI Diagnostic Radiology Projects Brain tumors Cervical spine fracture Brain age 14 chest abnormalities Brain abnormalities PE classification Pneumonia Coronary calcium scoring Brasfield scoring Renal cell carcinoma Lung cancer Normal/abnormal Fractures Bone age Renal abnormalities Knee cartilage Appendicitis Knee menisci and ligaments Deep vein thrombosis Fractures MR CT US XR PT AI Throughout The Imaging Life Cycle
Select precise treatment Enable patient self- with global reach scheduling
Disease Image classification protocoling
Prevent/detect disease Lower radiation dose early among populations Computer Image recon- and imaging time aided- struction and detection enhancement
Imaging Image triage sequence optimization
Decrease length of stay Image Answer the clinical question quality control
17 https://aimi.stanford.edu/ Reduce repeat imaging Infrastructure for AI in Medical Imaging
Clinical Practice Laboratory
DICOM Clinical EMR, RIS, Research Data Scanner Router PACS, Reporting Warehouse (STARR) Clinical Clinical Data images images Migration Cohort selection Image labels Cohort Management “Raw” Clinical Other Imaging • Role-based access detector images clinical Reporting data for AI for AI • Image viewing data System processing processing • Image annotation/ labeling AI results Machine-learning- AI AI ready data set results results Containerized UPSTREAM DOWNSTREAM Algorithm Algorithm Training & AI AI Prospective Evaluation and Deployment Platform Testing Vendor
18 Research Opportunities and Infrastructure Development Requirements for Translation
https://doi.org/10.1016/j.jacr.2019.04.014 • Data sets for training, testing, and validating AI algorithms • Standards for clinical integration of AI algorithms • Software use cases with common data elements • Balanced regulatory framework to ensure safety and efficacy Publicly-Released Labeled Radiology Datasets Publicly-Released Labeled Radiology Datasets
https://aimi.stanford.edu/ 21 QIBA = Reproducibility
Measure = 7 ± 26 Problem Solution
Variability in: Treat QIBA Profile specifying: • Patient handling • Calibration • Acqisition protocols • Patient Preparation • Image reconstruction • Acquisition Parameters • Segmentation ? • Reconstruction Parameters
• Image processing Wait • Resolution • Processing Parameters • Segmentation
Adapted from Kevin O’Donnell and Daniel Sullivan Thank You
Engage: @curtlanglotz @StanfordAIMI LinkedIn: aimi-stanford Join the mailing list: https://aimi.stanford.edu/engage/subscribe Apply to become affiliated faculty: https://aimi.stanford.edu/engage/affiliate-faculty