Ds3-Datascience-Polytechniqu…
Total Page:16
File Type:pdf, Size:1020Kb
Machine Learning for Personalised Healthcare: Advances and Challenges Danielle Belgrave Researcher Healthcare Machine Learning @DaniCMBelg My Research Patient 1 Patient 2 Patient 3 Patient 4 0 2 4 6 8 10 12 14 16 18 20 22 24 Time (hours) Latent Variable Modelling Longitudinal Data Analysis Missing Data X Y Z Patient-Centric Approach Causality Multidisciplinary Top 8 Challenges: DS in Healthcare 1. Address some of the technical challenges facing the community of machine learning for doing impactful healthcare research 2. Presentation of current solutions 3. Steps for future research Challenge # 1: Estimating Treatment Effects Intervention Population is split into 2 Outcomes for both groups by random allocation groups are measured Patient Group Control = Cured = Still Diseased Supervised Learning Intervention Control p1 = proportion in the intervention group who are cured p2 proportion in the control group who are cured H0: p1 - p2 = 0 e vs H1: p1 - p2 ≠ 0 Mean: p1 - p2 푝 (1−푝 ) 푝 (1−푝 ) = Cured = Not cured Variance: 1 1 + 2 2 푛1 푛2 푝 − 푝 Assume well-labelled groups Test Statistic: Z = 1 2 1 1 푝(1−푝)( + ) Machine recognises a new example 푛1 푛2 Classification, regression Challenge #2: Heterogeneous Populations Patient Group Same diagnosis same prescription Understanding Heterogeneity Drug NOT toxic Drug toxic but and beneficial Patient Group NOT beneficial Same diagnosis same prescription Drug toxic but Drug NOT toxic and beneficial NOT beneficial Supervised Learning Unsupervised Learning is not Enough Not cured Cured Patient Group Assume well-labelled groups Infer patterns/ discover underlying Machine recognises a new example data structure from a dataset without reference to labelled outcomes Classification, regression Clustering, Latent variable modelling Endotype Discovery to Understand Heterogeneity To identify subgroups of complex disease risk Treatment outcome explained by distinctive underlying mechanism Foundation of Stratified Medicine Seeking better-targeted interventions Accounting for Heterogeneity Individualised Healthcare Outcomes • Probabilistic Graphical Models • Reinforcement Learning • Deep Learning Understanding Heterogeneity Through Probabilistic Graphical Modelling Identifying Heterogeneous Patient Groups Parsimonious description of the data inferred from what is observed Probabilistic Programming Probabilistic reasoning system The evidence contains The probabilistic model specific information expresses general about a situation Probabilistic knowledge about a model situation The inference algorithm uses the Evidence model to answer Inference queries given evidence Algorithm Queries The answers to queries are framed as probabilities of Answer different outcomes The queries express the things that will help The basic components of a probabilistic reasoning system you make a decision Adapted from Pfeffer, Avi. "Practical probabilistic programming." International Conference on Inductive Logic Programming. Springer Berlin Heidelberg, 2010. Eco-systems for Probabilistic Programming Infer.NET Edward Observed values (data, priors) Probabilistic Pyro program Stan Infer.NET Inference Engine Algorith Infer.NET C# Algorith C# m compiler compiler m execution Probability distributions Understanding Heterogeneity Motivating Probabilistic Graphical Examples Modelling Example 1: Heterogeneity in Asthma Asthma Phenotypes: Observable Manifestations of Disease Asthma Medication Exacerbations Allergy Poor Wheeze Asthma Lung Symptoms Function Don’t Grow out of Respond to Asthma Late Severity Respond to Asthma treatment in Childhood treatment Endotypes? Different Diseases With Different Causes Causal Mechanisms of Asthma and Allergy Manchester Longitudinal birth cohort ~2000 children Progression of allergy: Eczema -> Asthma -> Rhinitis Symptoms Causally Linked Prevention strategy: Bristol Target children with eczema to reduce progression to asthma Longitudinal birth cohort and rhinitis ~10000 children Hidden Markov Model: “Allergic March” Eczema Age 1 Age 3 Age 5 Age 8 Age 11 Class Eczema State Eczema State Eczema State Eczema State Eczema State Wheeze Age 1 Age 3 Age 5 Age 8 Age 11 Class Wheeze State Wheeze State Wheeze State Wheeze State Wheeze State Rhinitis Age 1 Age 3 Age 5 Age 8 Age 11 Class Rhinitis State Rhinitis State Rhinitis State Rhinitis State Rhinitis State Children (n=9801) t e r A h e s s t h m n c a y Latent Class Disease a a n d u d M A l l e r g y S t Profile Longitudinal Latent Disease Profile Latent State Latent State Latent State Latent State Latent State Age 1 Age 3 Age 5 Age 8 Age 11 Class = 1,….,k Eczema Eczema Eczema Eczema Eczema Age 1 Age 3 Age 5 Age 8 Age 11 Wheeze Wheeze Wheeze Wheeze Wheeze Age 1 Age 3 Age 5 Age 8 Age 11 Rhinitis Rhinitis Rhinitis Rhinitis Rhinitis Age 1 Age 3 Age 5 Age 8 Age 11 Children (n=9801) Latent Class Disease Profile Disaggregating Symptom Heterogeneity The “Allergic March” reflects patterns at the population level, rather than the natural covariance of symptoms within individuals’ life courses Developmental profiles of are heterogeneous Danielle CM Belgrave, Raquel Granell, Angela Simpson, John Guiver, Christopher Bishop, Iain Buchan, A. John Henderson, and Adnan Custovic. Developmental Profiles of Eczema, Wheeze, and Rhinitis: Two Population-Based Birth Cohort Studies. PlosMedicine 2014. Example 2: Heterogeneity in Complex Chronic Diseases Scleroderma Aim: To predict a function of time that models the future trajectory of a single target clinical marker tracking a disease process of interest. Heterogeneity in Lung Function Schulam, Peter, and Suchi Saria. "Integrative analysis using coupled latent variable models for individualizing prognoses." The Journal of Machine Learning Research 17.1 (2016): 8244-8278. Probabilistic Models for Individualised Care Schulam, Peter, and Suchi Saria. "Integrative analysis using coupled latent variable models for individualizing prognoses." The Journal of Machine Learning Research 17.1 (2016): 8244-8278. Reinforcement Learning in Motivating Healthcare Examples Individualised Treatment Effects Task of Optimising Sequential Decision-Making Unobserved responses Observed decisions and response Unobserved Mechanical responses Ventilation? Sedate? Vasopressors Gottesman, O., Johansson, F., Komorowski, M., Faisal, A., Sontag, D., Doshi-Velez, F., & Celi, L. A. (2019). Guidelines for reinforcement learning in healthcare. Nature medicine, 25(1), 16-18. Reinforcement Learning in Healthcare What action maximises the reward Action (A) - State (S) - Reward (R) - Policy (π) - Value (V) Expected long-term return of the current state sunder policy π Vπ(s) Q-value or action-value (Q) Long-term return of the current state s, taking action a under policy π Qπ(s, a) v(s) = E[Rt+1 + λv(St+1)| St = s] 2 Qπ(s, a) = E[rt+1 + λrt+2 + λ rt+3 | s, a] = E[r + λ Qπ(s’, a’) | s, a] Q*π(s, a) = E[r + λmax Q*(s’, a’) | s, a] Successful Applications of RL in Healthcare 1. RL applied to optimizing antiretroviral therapy in HIV Parbhoo, S., Bogojeska, J., Zazzi, M., Roth, V. & Doshi-Velez, F. AMIA Summits on Translational Science Proceedings 2017, 239 (2017). 2. RL applied to tailoring antiepilepsy drugs for seizure control Guez, A., Vincent, R. D., Avoli, M. & Pineau, J. Treatment of epilepsy via batch- mode reinforcement learning. In Proceedings of the Twenty-Tird AAAI Conference on Artifcial Intelligence 1671–1678 (AAAI, 2008). 3. RL applied to interventions in ICU Komorowski, M., Celi, L. A., Badawi, O., Gordon, A. & Faisal, A. Nat. Med. 24, 1716– 1720 (2018). Confounding factors: Estimated effects of medication may be diminished if severely ill or high-risk patients are given a particular drug Effective sample size larger if learned policies are close to the clinician policies – RL better for refining existing practices rather than discovering new treatment approaches If the rewards we are trying to maximise are too simplistic, will the system behave as expected in real life Downfalls of RL in Healthcare Deep Learning Exceptionally effective at learning patterns of data Utilizes learning algorithms that derive meaning out of data by using a hierarchy of multiple layers that mimic the neural networks of our brain If you provide a system with tons of information, it begins to understand it and respond in useful ways Challenge #3: Heterogeneous Data Types Data “In-the-Wild” The Richer the Data Types, the more we need other methods Meta-challenges to Estimating Treatment Effect Heterogeneous Generalisability Causal Discovery Patient Populations Data “In-the-Wild” Missing Data Interpretability Challenge #4: Causality Causal Reasoning The questions that motivate most studies in the health, social and behavioral sciences are not associational but causal in nature. Before an association is assessed for the possibility that it is causal, other explanations such as chance, bias and confounding have to be excluded Require some knowledge of the data-generating process - cannot be computed from the data alone, nor from distributions governing data Aim: to infer dynamics of beliefs under changing conditions, for example, changes induced by treatments or external interventions. Pearl, Judea. "Causal inference in statistics: An overview." Statistics surveys 3 (2009): 96-146. Efficacy and mechanism evaluation: Causal Framework for Personalising Health U Mediator Predictive biomarker (moderator) Outcomes Random Allocation Prognostic biomarker (risk factor) Example: Personalisation of Cancer Treatment U Tumor Size Genetic Marker Outcome Treatment (Survival) Prognostic biomarker