Multi-Objective Molecule Generation Using Interpretable Substructures
Total Page:16
File Type:pdf, Size:1020Kb
Multi-Objective Molecule Generation using Interpretable Substructures Wengong Jin 1 Regina Barzilay 1 Tommi Jaakkola 1 Abstract with substructures (e.g., functional groups or later pieces) Drug discovery aims to find novel compounds that drive specific properties of interest, and then combine with specified chemical property profiles. In terms these building blocks into a target molecule. To automate of generative modeling, the goal is to learn to this process, our model has to learn two complementary sample molecules in the intersection of multiple tasks: (1) identification of the building blocks that we call property constraints. This task becomes increas- rationales, and (2) assembling multiple rationales together ingly challenging when there are many property into a fully formed target molecule. In contrast to competing constraints. We propose to offset this complex- methods, our generative model does not build molecules ity by composing molecules from a vocabulary from scratch, but instead assembles them from automat- of substructures that we call molecular rationales. ically extracted rationales already implicated for specific These rationales are identified from molecules as properties (see Figure1). substructures that are likely responsible for each We implement this idea using a generative model of property of interest. We then learn to expand ratio- molecules where the rationale choices play the role of latent nales into a full molecule using graph generative variables. Specifically, a molecular graph G is generated models. Our final generative model composes from underlying rationale sets S according to: molecules as mixtures of multiple rationale com- X pletions, and this mixture is fine-tuned to preserve P (G) = P (GjS)P (S) (1) S the properties of interest. We evaluate our model on various drug design tasks and demonstrate sig- As ground truth rationales (e.g., functional groups or sub- nificant improvements over state-of-the-art base- graphs) are not provided, the model has to extract candidate lines in terms of accuracy, diversity, and novelty rationales from molecules with the help of a property pre- of generated compounds. dictor. We formulate this task as a discrete optimization problem efficiently solved by Monte Carlo tree search. Our rationale conditioned graph generator, P (GjS), is initially 1. Introduction trained on a large collection of real molecules so that it is capable of expanding any subgraph into a full molecule. The key challenge in drug discovery is to find molecules The mixture model is then fine-tuned using reinforcement that satisfy multiple constraints, from potency, safety, to learning to ensure that the generated molecules preserve all desired metabolic profiles. Optimizing these constraints the properties of interest. This training paradigm enables us simultaneously is challenging for existing computational to realize molecules that satisfy multiple constraints without models. The primary difficulty lies in the lack of training observing any such instances in the training set. arXiv:2002.03244v3 [cs.LG] 2 Jul 2020 instances of molecules that conform to all the constraints. For example, for this reason, Jin et al.(2019a) reports over The proposed model is evaluated on molecule design tasks 1 60% performance loss when moving beyond the single- under different combinations of property constraints. Our constraint setting. baselines include state-of-the-art molecule generation meth- ods (Olivecrona et al., 2017; You et al., 2018a). Across all In this paper, we propose a novel approach to multi- tasks, our model achieve state-of-the art results in terms of property molecular optimization. Our strategy is inspired accuracy, novelty and diversity of generated compounds. In by fragment-based drug discovery (Murray & Rees, 2009) particular, we outperform the best baseline with 38% abso- often followed by medicinal chemists. The idea is to start lute improvement in the task with three property constraints. We further provide ablation studies to validate the benefit 1MIT CSAIL. Correspondence to: Wengong Jin <wen- [email protected]>. of our architecture in the low-resource scenario. Finally, we show that identified rationales are chemically meaningful in Proceedings of the 37 th International Conference on Machine a toxicity prediction task (Sushko et al., 2012). Learning, Vienna, Austria, PMLR 119, 2020. Copyright 2020 by the author(s). 1https://github.com/wengong-jin/multiobj-rationale Multi-Objective Molecule Generation using Interpretable Substructures Attribution GSK3β Rationales JNK3 Rationales 100% 100% N H N 1 H N N HN HN 98.5% 97.4% H active HN H N N N 74.8% N N N 0.75 N N NH2 N N N Rationale N N N NH N H NH N 2 OH GSK3β Predictor 0.5 Rationales as 47.9% “macro-actions” Attribution 0.25 Success rate H HN N N HN H H REINVENT RationaleRL N N H N N N HN N Nactive Generated H N N N N 1 2 4 N NH dual inhibitorsN N NH Rationale O JNK3 Predictor N 2 OH Number of Constraints Figure 1. Illustration of RationaleRL. Left: To generate a dual inhibitor against biological targets GSK3β and JNK3, our model first identifies rationale substructures S for each property. Note that rationales are not provided as domain knowledge. Middle: The model learns to compose multiple rationales S into a complete molecule G. Right: Our method achieves much higher success rate than the current state-of-the-art molecule design method REINVENT (Olivecrona et al., 2017)) under four property constraints. 2. Related Work Samanta et al., 2018; Liu et al., 2018; De Cao & Kipf, 2018; Ma et al., 2018; Seff et al., 2019). On the optimization Reinforcement Learning One of the prevailing paradigms side, the task has been formulated as reinforcement learn- for drug design is reinforcement learning (RL) (You et al., ing (Guimaraes et al., 2017; Olivecrona et al., 2017; Popova 2018a; Olivecrona et al., 2017; Popova et al., 2018), which et al., 2018; You et al., 2018a; Zhou et al., 2018), continu- seeks to maximize the expected reward defined as the sum ous optimization in the latent space learned by variational of predicted property scores using the property predictors. autoencoders (Gomez-Bombarelli´ et al., 2018; Kusner et al., Their approach learns a distribution P (G) (a neural network) 2017; Dai et al., 2018; Jin et al., 2018; Kajino, 2018; Liu for generating molecules. Ideally, the model should achieve et al., 2018), or graph-to-graph translation (Jin et al., 2019b). high success rate in generating molecules that meet all the In contrast to existing approaches, our model focuses on the constraints, while maintaining the diversity of P (G). multi-objective setting of the problem and offers a different The main challenge of RL is the reward sparsity, especially formulation for molecule generation based on rationales. when there are multiple competing constraints. For illus- Interpretability Our rationale based generative model tration, we test a state-of-the-art reinforcement learning seeks to provide transparency (Doshi-Velez & Kim, 2017) method (Olivecrona et al., 2017) under four property con- for molecular design. The choice of rationales P (S) is visi- straints: biological activity to GSK3β, JNK3, drug-likeness ble to users and can be easily controlled by human experts. and synthetic accessibility (Li et al., 2018). As shown in Prior work on interpretability primarily focuses on finding Figure1, initially the success rate and diversity is high when rationales (i.e., explanations) of model predictions in image given only one of the constraints, but they decrease dra- and text classification (Lei et al., 2016; Ribeiro et al., 2016; matically when all the property constraints are added. The Sundararajan et al., 2017) and molecule property predic- reason of this failure is that the property predictor (i.e., re- tion (McCloskey et al., 2019; Ying et al., 2019; Lee et al., ward function) remains black-box and the model has limited 2019). In contrast, our model uses rationales as building understanding of why certain molecules are desirable. blocks for molecule generation. Our framework offsets this complexity by understanding property landscape through rationales. At a high level, the 3. Proposed Approach: RationaleRL rationales are analogous to options (Sutton et al., 1999; Stolle & Precup, 2002), which are macro-actions leading Molecules are represented as graphs G = (V; E) with atoms the agent faster to its goal. The rationales are automatically V as nodes and bonds E as edges. The goal of drug discov- discovered from molecules with labeled properties. ery is to find novel compounds satisfying given property constraints (e.g., drug-likeness, binding affinity, etc.). With- Molecule Generation Previous work have adopted var- out loss of generality, we assume the property constraints to ious approaches for generating molecules under specific be of the following form: property constraints. Roughly speaking, existing methods can be divided along two axes — representation and opti- Find molecules G mization. On the representation side, they either operate (2) Subject to r (G) ≥ δ ; i = 1; ··· ;M on SMILES strings (Gomez-Bombarelli´ et al., 2018; Segler i i et al., 2017; Kang & Cho, 2018) or directly on molecular For each property i, the property score ri(G) 2 [0; 1] of graphs (Simonovsky & Komodakis, 2018; Jin et al., 2018; molecule G must be higher than threshold δi 2 [0; 1].A Multi-Objective Molecule Generation using Interpretable Substructures Rationales S1 Multi-Property Generated Property Property 1 H Rationales S compound G Predictors N N HN Graph completion Dataset MCTS H HN H N N N model H pos N N H P(G|S) N N D N N N 1 N NH N Sample rationaleN NH N 2 OH N N S ∼ P(S) Rationales 2 NH S NH N Property 2 N MCTS HN Dataset N N N Dpos N NH2 H 2 HO N Rationale Extraction Molecule Generation Figure 2. Overview of our approach.