Neural message passing for joint paratope-epitope prediction Alice Del Vecchio 1 Andreea Deac 2 3 4 Pietro Lio` 1 Petar Velickoviˇ c´ 4 Abstract Hence, both the antibody and the antigen may be viewed as sequences of amino acid residues. Their binding site Antibodies are proteins in the immune system consists of two regions: the paratope on the antibody, and which bind to antigens to detect and neutralise the epitope on its corresponding antigen. Predicting them them. The binding sites in an antibody-antigen can therefore be posed as a binary classification problem: interaction are known as the paratope and epitope, for each amino acid residue in the antibody and antigen, respectively, and the prediction of these regions respectively, do they participate in the binding? is key to vaccine and synthetic antibody develop- ment. Contrary to prior art, we argue that paratope However, proteins can also be considered as graphs with its and epitope predictors require asymmetric treat- residues as nodes, with two nodes sharing an edge if their ment, and propose distinct neural message passing residues are spatially close. Recently, such contact graphs architectures that are geared towards the specific have been directly leveraged for protein function prediction aspects of paratope and epitope prediction, re- by Gligorijevic et al.(2020). spectively. We obtain significant improvements The advantage of considering a sequence based approach on both tasks, setting the new state-of-the-art and over a graph-based approach is that structural information recovering favourable qualitative predictions on is much harder to obtain. However, recent advancements antigens of relevance to COVID-19. in single-protein structure prediction from AlphaFold 2 (Jumper et al., 2020) have comparable accuracy to experi- mental methods. This makes the incorporation of structure 1. Introduction justifiable as obtaining this information is becoming easier. Antibody binding site prediction is an important unsolved In this work we study the joint epitope-paratope prediction problem in immunology and is a prerequisite for in silico task in detail, identify an inherent asymmetry between the vaccine and synthetic antibody design. Currently, the most two tasks, and propose epitope-paratope message passing accurate method for determining these regions is by observ- (EPMP), a hybrid method that explicity leverages this asym- ing the residues which are spatially close in the 3D bound metry to design potent predictors for both tasks. We set the structure, obtainable by experimental techniques such as new state-of-the-art on both tasks, and illustrate qualitatively X-ray crystallography (Smyth & Martin, 2000). However, meaningful predictions on antigens of present importance. these experimental methods are time consuming and expen- sive, so there is a need to develop computational methods which can overcome these problems to aid faster develop- 2. Related work ment of therapeutics. For example, in the ongoing COVID- Of the two tasks, paratope prediction has seen much more 19 pandemic, mutations in the antigen have been shown to success. This is because the paratope is necessarily tightly cause changes to the binding mechanism (Thomson et al., localised on the tips of the Y-shaped antibody (the so-called arXiv:2106.00757v2 [q-bio.QM] 25 Jul 2021 2021), potentially impacting the efficacy of existing thera- Fv region). In fact, many early works on paratope prediction peutics. Such a computational tool would allow for quick (e.g. Liberis et al.(2018)) consider an even narrower region, determination of the impact of these mutations, allowing the complementarity-determining regions (CDRs)—which for therapeutics development to keep up with fast occurring combined rarely exceeds 180 residues. This allowed for mutants. discarding most of the antibody sequence, leading to a better- We only consider scenarios where the antigen is a protein. balanced prediction problem. In rare cases, the paratope can be located outside of the CDRs; accordingly, we will utilise 1 2 3 University of Cambridge Mila Universite´ de Montreal´ the entire Fv region as input. 4DeepMind. Correspondence to: Alice Del Vecchio <[email protected]>. Paratope prediction was tackled both by sequential (Liberis et al., 2018; Deac et al., 2019; Honda et al., 2020) and The 2021 ICML Workshop on Computational Biology. Copyright structural (Krawczyk et al., 2013; Daberdaku & Ferrari, by the author(s). Neural message passing for joint paratope-epitope prediction 2019; Pittala & Bailey-Kellogg, 2020) approaches in prior in their respective datasets. art. The sequence-based models offer a strong and fast As in other works, the paratope is defined to include the baseline in this setting, as the median gap between paratope residues on the antibody which are less than 4.5A˚ away residues in a CDR is 1–2 residues (Akbar et al., 2021)— from any heavy atom in the antigen (non hydrogen atoms). hence the paratope is reasonably sequentially located within The epitope is defined similarly, but for antigen residues. the CDR. The paratope dataset is a subset of the AbDb database (Fer- On the other hand, there is no Fv-like region for epitopes, dous & Martin, 2018), containing 308 complexes in the and as a result the entire antigen needs to be considered. training and validation sets and 152 complexes in the test This gives epitope predictions a high degree of class imbal- set. The Fv regions of the antibodies are used. In the test ance: as no part of the antigen can be easily discarded at set, 9.4% of antibody residues are part of the positive class. the input stage, most antigen residues will not belong to an epitope. Further, epitope residues may be far apart in the The epitope dataset is a subset of the SAbDab database (Dun- antigen sequence, implying that sequential epitope models bar et al., 2014) and the Docking Benchmark v5 (Vreven are likely to be weak baselines. Epitopes are also specific et al., 2015). There are 132 complexes in the combined to the antibody that targets them, implying that antibody in- training/validation set, and 30 complexes in the test set. The formation needs to be carefully leveraged within an epitope CDR regions of the antibodies are used, to provide highly predictor (Krawczyk et al., 2014; Andersen et al., 2006). targeted context. In the test set, 7.8% of antigen residues were part of the positive class. 3. Present work The same features as in PECAN are used: a 62-dimensional feature vector for each residue, containing a one-hot encod- Clear asymmetry arises between paratope and epitope pre- ing of the amino acid, alignment based features from the diction: paratopes are highly sequential and can be predicted Position Specific Scoring Matrix (PSSM), surface accessi- well in isolation, while epitopes are structural in nature and bility scores and a profile counting the number of amino are inherently conditioned by the paratope. Hence, we con- acid types in a sphere of radius 8A˚ around the residue. The clude that a joint paratope-epitope predictor should ideally graphs in PECAN are restricted to a neighbourhood of the have separately tuned architectures for the two tasks. How- closest 15 neighbours (up to a distance of 10A).˚ To allow ever, the present state-of-the-art for joint paratope-epitope these to be used as undirected graphs, we symmetrise the ad- prediction, PECAN (Pittala & Bailey-Kellogg, 2020), uses jacency matrices. This diversifies the structure in the graph, an entirely symmetric architecture for both. as residue nodes will have varying numbers of neighbours. Accordingly, here we propose epitope-paratope message passing (EPMP), which designs highly asymmetrical neu- 5. Methods ral networks for paratope (Para-EPMP) and epitope (Epi- EPMP) predictors. We now present our two predictors within EPMP, by consid- ering paratope and epitope prediction in turn. Para-EPMP is designed to exploit the sequentiality of the paratope (using a` trous convolutional neural networks), while also leveraging structural cues with graph neural net- 5.1. Paratope model (Para-EPMP) works (GNNs). We use the most expressive class of spatial For our paratope model, the input features are initially pro- GNN architectures to further take advantage of the rich and cessed sequentially, with graph structure incorporated later. balanced paratope label information. This model and its hyperparameters are outlined in Figure Conversely, Epi-EPMP is purely structural, and extensively 1, and closely echoes our previous discussion. enforces exploiting the contextual cues from the antibody, The residue features are first passed through three a` trous along with a multi-task approach that ensures the model’s CNNs (Kalchbrenner et al., 2016), as used in the sequence- predictions are adequately regularised. Given the high poten- based paratope predictor AG-Fast-Parapred (Deac et al., tial for overfitting, we leverage the more versatile attentional 2019), as well as the recent Tail-GNN model (Spalevic´ et al., class of GNNs. 2020). A` trous CNNs have a dilation term which creates gaps in the convolution kernel, allowing the entire sequence 4. Data to be covered in fewer layers, preventing overfitting. The preprocessed datasets are taken from PECAN (Pittala The output embeddings from the CNN are then passed & Bailey-Kellogg, 2020). Different datasets are used for through two message passing neural network (MPNN) lay- paratope and epitope prediction, as complexes have been ers (Gilmer et al., 2017) over the proximity-based residue filtered to ensure that no two antibodies/antigens are similar graph. These layers have single linear layers as their mes- Neural message passing for joint paratope-epitope prediction sage and update functions and use the sum aggregation. antibody-antigen boundary. For this purpose, we used a MPNNs correspond to the most expressive class of spatial graph attention network (GAT) (Velickoviˇ c´ et al., 2018) GNNs, making them highly appropriate for the kind of data over the fully connected bipartite graph between the antigen available for the paratope task.