Medical Imaging and AI Meets Genomics

The new Bayer LifeHub UK Medical Imaging and AI meets Genomics

///////////

Guenther Brueggenwerth Bayer LifeHub UK Lead Radiology

[email protected] Linkedin: LifeHub UK +- UK ABPI PP-OTH-GB-0348 / Jan 2020

London, 30 Jan 2020 Festival of Genomics

THIS IS AN AGE OF DISRUPTION FOR PHARMA, DRIVEN BY SCIENTIFIC ADVANCEMENTS, PROGRESS IN DATA SCIENCE, AND NON-TRADITIONAL ENTRANTS

Changing role of patient, DATA / ANALYTICS / AI provider, and payors

Advanced Pricing diagnostics Artificial transparency New players reshaping intelligence ecosystem

NEW BUSINESS MODELS PROMISE OF SCIENCE Outcomes based Cell and value gene therapy Mobile diagnostics

Healthy Connected Regenerative ageing patients medicine therapies

TYPICAL CHALLENGES OF LARGE PHARMA (vs BIOTECH) INCLUDE: LOSS OF EXCLUSIVITY, PIPELINE, AND PROCESS-ORIENTATED CULTURE 2 /// AI @ the new Bayer LifeHub UK /// Jan 2020

1 Global challenges in Life Sciences are too big to tackle alone: Ecosystems can boost the innovation capability of its community

Closed Innovation Open Innovation Innovation Ecosystems

Internally focused Externally focused Ecosystem focused

3 /// AI @ the new Bayer LifeHub UK /// Jan 2020

Bayer newly-designed global network of LifeHubs to advance Innovation

Opened October 2019 at Bayer UK Headquarter, Reading

4 /// AI @ the new Bayer LifeHub UK /// Jan 2020

2 LifeHub UK is focusing on AI imaging solutions to optimize drug discovery and disease diagnosis/therapy

UK is the third biggest market in the world for AI investment 1

UK has an ambitious UK has a unique agenda around digital healthcare system and AI

1 Tech Nation 2019

5 /// AI @ the new Bayer LifeHub UK /// Jan 2020

Connecting with the community through networking, mentoring and meetups, with open & transparent discussions about leading AI 1 applications and other digital technologies within healthcare

Exploring impactful outcomes by collaborating with customers, entrepreneurs and 2 innovators within the local community while building new skills and business models

Co-creating solutions with interactive workshops on challenges blending 3 Bayer & UK brainpower

Partnering with experts through alliances that develop cutting edge imaging AI 4 solutions while leveraging the network and reach of a global company → Focus areas Radiology, Oncology, Cardiovascular, Pulmonology, Ophthalmology, WH

6 /// AI @ the new Bayer LifeHub UK /// Jan 2020

3 Our ambition To connect with the brightest minds to jointly explore, discover and develop leading customer-centric solutions in Life Sciences Our team Leading experts imaging data scientists and data engineers, radiologists, radiographers, program managers and marketing professionals Our partners • Sensyne Health • Scouting for further clinical and industry partners

7 ////// AIAI @@ thethe newnew BayerBayer LifeHubLifeHub UK UK ////// JanJan 20202020

Radiomics

Processing and analysis of radiological imaging data beyond visibility of the naked eye (→ agnostic features) and classical morphology criteria (→ semantic features)

Image acquisition CT, MRI, PET etc Identification and segmentation of the volumes of interest in a tumor Automated or semi-automated methods more accurate than manual methods Multiple qualitative and quantitative features are generated from raw data over the region of interest Morphological, statistical, regional, model-based Can be e.g. reduction from > 1000 features to < 15 Combining information from multiple imaging modalities can provide a multispectral view of the tumor and allows improved tumor characterization Discovering relationships among radiomic features and “-omics”, pathology/histology, and clinical data

8 /// AI @ the new Bayer LifeHub UK /// Jan 2020

4 Radiomics + Genomics = Radiogenomics

Radiogenomics aims to correlate imaging characteristics (ie, the imaging phenotype) with gene expression patterns, gene mutations, and other genome‐related characteristics

Radiomics and multi-omic flowchart for GBM IDH1 wild type brain tumor with contrast enhanced magnetic resonance imaging Integration of Radiomic and Multi-omic Analyses Predicts Survival of Newly Diagnosed IDH1 Wild-Type Glioblastoma, A. Chaddad, P. Daniel, S. Sabri, C. Desrosiers, B. Abdulkarim Cancers 2019, 11(8), 1148; https://doi.org/10.3390/cancers11081148 http://creativecommons.org/licenses/by/4.0/

9 /// AI @ the new Bayer LifeHub UK /// Jan 2020

Radiogenomics Indications

Current radiomic and radiogenomic studies are limited to few types of common cancers

Glioblastoma multiforme Non-small cell lung cancer Hepatocellular carcinoma Intrahepatic cholangiocarcinoma Breast cancer Prostate cancer Renal cell carcinoma Cervical cancer Ovarian cancer

Lo Gullo, R., Daimiel, I., Morris, E.A. et al. Combining molecular and imaging metrics in cancer: radiogenomics. Insights Imaging 11, 1 (2020) doi:10.1186/s13244-019-0795-6 https://rdcu.be/b0AOk https://creativecommons.org/licenses/by/4.0/

10 /// AI @ the new Bayer LifeHub UK /// Jan 2020

5 Radiogenomics Analysis Example Prostate Ca: Significant correlations between genes and quantitative imaging features, indicating the presence of prognostic signal in the radiomic features

Pearson’s correlation analysis of imaging features and 65 genes from commercially available prostate cancer classifiers Stoyanova R., Pollack A., Takhar M., Lynne C., Parra N., Lam L. L.C., Alshalalfa M., Buerki C., Castillo R., Jorda M., Ashab H., Kryvenko O. N., Punnen S., et al Association of multiparametric MRI quantitative imaging features with prostate cancer gene expression in MRI-targeted prostate biopsies. Oncotarget. 2016; 7: 53362-53376. http://www.oncotarget.com/index.php?journal=oncotarget&page=article&op=view&path%5B%5D=10523&path%5B%5D=43017 http://creativecommons.org/licenses/by/3.0/

11 /// AI @ the new Bayer LifeHub UK /// Jan 2020

Radiogenomics Challenges

The implementation of radiogenomic in clinical practice is still not routinely done

Several limitations associated with radiogenomic analysis Gene expression and signaling pathways are extremely complex Difficult to match the large amount of data from whole-genome sequencing with imaging data The dimensionality of genomic data should be reduced to match that obtained from imaging studies Differences in quantitative imaging features are not only related to gene expression Can also be related to other factors such as patient characteristics or imaging technique Inter- and intra-institutional heterogeneity of datasets due to different hardware and scan protocols limits the generalizability of results The limitations associated with interobserver variation make qualitative imaging features less preferable Often-small patient cohort and only retrospective radiogenomic studies

12 /// AI @ the new Bayer LifeHub UK /// Jan 2020

6 Radiogenomics Opportunities

Radiogenomics can play an important role in providing accurate imaging surrogates which are correlated with genetic expression

In future radiogenomics might serve as a selection guide for confirmatory genetic testing Radiomics and radiogenomics are promising to increase precision in Diagnosis, assessment of prognosis, and prediction of treatment response Providing valuable information for patient care throughout the course of the disease, given that this information is easily obtainable with imaging With personal medicine playing an increasing role in clinical practice, radiogenomics in particular can allow faster and more efficient genotype identification and be applied to all cancer types Larger prospective studies and standardization will be needed to define relevant imaging biomarkers before they can be implemented into the clinical workflow Deep learning models like convolutional neural networks (CNNs) and pattern-recognition algorithms like random forests and elastic net can also be used to learn an optimal representation directly from the images Training CNNs etc. requires large datasets of images Lo Gullo, R., Daimiel, I., Morris, E.A. et al. Combining molecular and imaging metrics in cancer: radiogenomics. Insights Imaging 11, 1 (2020) doi:10.1186/s13244-019-0795-6 https://rdcu.be/b0AOk https://creativecommons.org/licenses/by/4.0/ 13 /// AI @ the new Bayer LifeHub UK /// Jan 2020

In the following Arndt Schmitz will present a case study where AI is being applied to pathology ///////////

7 Clinical Operations Bayer AG, Berlin, Germany Histogenomics Case Study

/////////// Arndt Schmitz Festival of Genomics London, Jan 2020

Personalized medicine Lung cancer as a case study

Lung Cancer is a TOP4 cancer indication.

Each year, in the USA alone, approx. 175,000 new cases are diagnosed.

The individual cases are triggered by activation of oncogenes, with varying frequency.

New molecular therapeutics are effective for cases affected by one specific oncogene.

=> Some treatments target a few thousand cases. => How to identify the patients who might benefit?

16 /// Schmitz /// Histogenomics Case Study /// 2020

8 Medical Imaging – radiology vs pathology Major data source, complementary to eHR & lab tests

Radiology Pathology imaging in vivo ex vivo invasive no/ minimally minimally/ yes image generation typically digital typically analogue & scan image resolution millimeterscale micrometer scale image varieties scanner settings, chemical dyes, contrast media immunohisto- etc. chemistry etc. ‚digital readiness‘ more mature lessmature use of AI emerging emerging

Modified from University of Virginia Health System

17 /// Schmitz /// Histogenomics Case Study /// 2020

The place of AI tools in the patient journey after cancer diagnosis. Problem statement and vision for increased value of medical images

Diagnosis: EHR data fields Cancer Output HE images of tumor AI Estimated Symptoms Radiology images probability cancer is driven by XYZ gene

Feedback to AI for continuous improvement

CAP / CLIA Lab Physician Physician orders performs IVD test, adjusts molecular confirms XYZ gene treatment XYZ gene test aberration

18 /// Schmitz /// Histogenomics Case Study /// 2020

9 Bayer is exploring an AI pre-screening tool based on HE images.

Solution: Create AI algorithm to Aim: Increase oncogene Some cancer types have a low % preselect based on HE imaging diagnosis rate through of patients with XYZ oncogene data prioritization

Testing all solid tumor patients for HE diagnosis is standard of care and Finally physicians do not need to test these oncogenes is a challenge these images will provide the basis for every patients but only a small training an AI algorithm in combination subgroup which saves time and cost with EHR data

/// Schmitz /// Histogenomics Case Study /// 2020

What are HE images, actually?

Solution: Create AI algorithm to Deep learning & HE diagnosis preselect based on HE imaging medical imaging data

• When the diagnosis of a solid HE diagnosis is standard of care Reasons for this strategy include cancer is suspected, a tissue biopsy and is taken. these images will provide the basis for • Recent advancements in • The tissue is sliced ultrathin and training an AI algorithm in combination AI image recognition algorithms stained with two dyes, hematoxylin with EHR data • Recent increases in GPUs’ power and eosin. These color nuclei of and availability cells and most proteins in the cytoplasm. • Tumorigenesis results in changes in • A trained pathologist can then look tissue morphology at the shape of the cells and their • Others recently used such nuclei, and the arrangement of the approaches with some success: cells in the tissue. • Lung Cancer classifier [1] • This will diagnose the solid tumor. • Colorectal Cancer microsatellite • HE is the most widely used stain in instability classifier [2] medical diagnosis.

1Classification and mutation prediction from non-small cell lung cancer histopathology images using deep learning, Choudray et al., PMID 30224757 2Deep learning can predict microsatellite instability directly from histology in gastrointestinal cancer, Kather et al., PMID 31160815

/// Schmitz /// Histogenomics Case Study /// 2020

10 Machine learning enabled agents: the Future for Medical Imaging

Example: Cancer Classification Using Convolutional Neuronal Network (CNN) Approach

Cancer Cohort Select Pre-processing . CNN (Convolutional Neural Network) to Dataset Cases classify pathology images in order to classify cancer cases. Enhanced Image . Inputs: H&E stained tumor specimens labelled with molecular annotation Feature Extraction . Training: Train CNN using this data, driving oncogenes as classes. Machine Learning Architecture Different Classes . Validation: Compare results with previously unseen images with the wet lab results from molecular tests, Input Layer Hidden Layers Output Layer preferred in vitro diagnostics. Oncogene X or Y Inference Modified from AAIH.org /// Schmitz /// Histogenomics Case Study /// 2020

Narrowing the search: AI pre-screening tool based on HE images The algorithm consists of a convolutional neural network which learns the distinctive features of the oncogene positive vs. negative HE tiles. To train such network we provided more than 100 000 tiles extracted from the tumor regions of the H&E slides. 1. Training

Deep Training TumorTumor Learning set oncogeneTRK fusion XYZ >1000 tiles per algorithm POS. tumorpositive HE slide

All tiles split 2. Validation Trained Test set Deep Other Learning OtherOther algorithm oncogene XYZTumorTumor NEG. tumorTumor >1000 tiles per HE slide

Prediction: case Per tile score and pos / neg accuracy

/// Schmitz /// Histogenomics Case Study /// 2020

11 In the next stage, the AI algorithm is leveraging the results from the tile level to the patient level. AI Solution Tissue HE slide with an The tumor region of the annotated tumor region slide is split into tiles I. Deep Learning II. Slide level Tiles Classification prediction Tumor Convolutional neural Calculates probability of network calculates the the entire slide to be >1000 tiles per score of each tile to be oncogene positive using HE slide oncogene positive tile-level predictions

Data preparation X%

If the predicted probability is higher than predefined threshold, then the slide is Heat map generated based on considered to be the tile level scores (brown oncogene positive corresponds to high probability /// Schmitz /// Histogenomics Case Study /// 2020 of oncogene being the driver) 23

What could the value of an AI tool be? • Economics of artificial intelligence in medical imaging / molecular testing

• Shift from consumables testing per case towards upfront capital investment in digital infrastructure Deep learning can predict microsatellite instability directly from histology in gastrointestinal cancer Jakob 25, pages1054–1056(2019)Nikolas Kather et al Nature Medicinevolume 24 /// Schmitz /// Histogenomics Case Study /// 2020

12 Interdisciplinary collaboration in “Translational Data Medicine”

Pathology/Radiology Molecular Assays Partnering Internal partnerships Data Science Image generation Assays for oncogene Provide data Functions from Legal via AI algorithm development Image annotation activation status : as training / validation business consulting to Evaluation of emerging Disease understanding ‘ground truth’ annotation material from outside procurement etc. technologies such as  University Hospitals  Diverse partners: partners  Internal alignment & federated learning internal, Contract labs,  Diverse partners: tremendous support  Internal together with university hospitals clinics, startups etc. selected external partners

/// Schmitz /// Histogenomics Case Study /// 2020

Long term vision: genomics, radiomics, pathology  building together new diagnostic opportunities AI image recognition algorithms trained on genomics annotated image sets might streamline routine tumor genetic testing

confirmatory genetic testing allowing effective decisions

Example: EGFR in lung cancer Example: EGFR in lung cancer genotype prediction via genotype prediction via radiology images & AI pathology images & AI e.g. XY Li, PMID 30746208 e.g. Choudray, PMID 30224757 radiomics histomics

Foundation: genomics- annotated medical images

/// Schmitz /// Histogenomics Case Study /// 2020

13 Arndt Schmitz, Clinical Sciences Operations Berlin Thank you!

/////////// Questions?

/// Schmitz /// Histogenomics Case Study /// 2020

Backup – ETHICS GUIDELINES FOR TRUSTWORTHY AI HIGH-LEVEL EXPERT GROUP of the European Commission, April 2019

Trustworthy AI & Explainable AI:

For a system to be trustworthy, we must be able to understand why it behaved a certain way and why it provided a given interpretation.

A whole field of research, Explainable AI (XAI) tries to address this issue to better understand the system’s underlying mechanisms.

For example, backpropagation is one technique to visualize which parts of an image are weighted higher or lower by a CNN.

/// Schmitz /// Histogenomics Case Study /// 2020

14 Backup - Data transfer vs. data access – federated learning as a solution?

External data transferred to Bayer: same situation as Bayer data preferred

Bayer External data accessible by Bayer, data only in partner‘s IT environment: requiring ‚federated learning‘ not preferred, technically challenging

External data not accessible to Bayer, available in partner‘s IT environment: requiring ‚outsourcing‘ / control of results

modified from https://www.intel.ai/federated-learning-for-medical-imaging

/// Schmitz /// Histogenomics Case Study /// 2020

Backup - How to regulate an algorithm that is constantly changing? FDA proposal 2019: Total Product Lifecycle (TPLC) Regulatory Approach

/// Schmitz /// Histogenomics Case Study /// 2020