Journal of Computer-Aided Molecular Design (2020) 34:335–370 https://doi.org/10.1007/s10822-020-00295-0 Assessing the accuracy of octanol–water partition coefcient predictions in the SAMPL6 Part II log P Challenge Mehtap Işık1,2 · Teresa Danielle Bergazin3 · Thomas Fox5 · Andrea Rizzi1,6 · John D. Chodera1 · David L. Mobley3,4 Received: 7 November 2019 / Accepted: 24 January 2020 / Published online: 27 February 2020 © Springer Nature Switzerland AG 2020 Abstract The SAMPL Challenges aim to focus the biomolecular and physical modeling community on issues that limit the accuracy of predictive modeling of protein-ligand binding for rational drug design. In the SAMPL5 log D Challenge, designed to benchmark the accuracy of methods for predicting drug-like small molecule transfer free energies from aqueous to nonpolar phases, participants found it difcult to make accurate predictions due to the complexity of protonation state issues. In the SAMPL6 log P Challenge, we asked participants to make blind predictions of the octanol–water partition coefcients of neutral species of 11 compounds and assessed how well these methods performed absent the complication of protonation state K K efects. This challenge builds on the SAMPL6 p a Challenge, which asked participants to predict p a values of a superset of the compounds considered in this log P challenge. Blind prediction sets of 91 prediction methods were collected from 27 research groups, spanning a variety of quantum mechanics (QM) or molecular mechanics (MM)-based physical meth- ods, knowledge-based empirical methods, and mixed approaches. There was a 50% increase in the number of participating groups and a 20% increase in the number of submissions compared to the SAMPL5 log D Challenge. Overall, the accuracy of octanol–water log P predictions in SAMPL6 Challenge was higher than cyclohexane–water log D predictions in SAMPL5, likely because modeling only the neutral species was necessary for log P and several categories of method benefted from the vast amounts of experimental octanol–water log P data. There were many highly accurate methods: 10 diverse methods achieved RMSE less than 0.5 log P units. These included QM-based methods, empirical methods, and mixed methods with physical modeling supported with empirical corrections. A comparison of physical modeling methods showed that QM-based methods outperformed MM-based methods. The average RMSE of the most accurate fve MM-based, QM-based, empirical, and mixed approach methods based on RMSE were 0.92 ± 0.13, 0.48 ± 0.06, 0.47 ± 0.05, and 0.50 ± 0.06, respectively. Keywords Octanol–water partition coefcient · log P · Blind prediction challenge · SAMPL · Free energy calculations · Solvation modeling Abbreviations log D log10 of organic solvent-water distribution coef- SAMPL Statistical Assessment of the Modeling of Pro- fcient ( Dow) teins and Ligands pKa − log10 of the acid dissociation equilibrium log P log10 of the organic solvent-water partition coef- constant fcient ( Kow ) of neutral species SEM Standard error of the mean RMSE Root mean squared error MAE Mean absolute error Mehtap Işık and Teresa Danielle Bergazin have contributed Kendall’s rank correlation coefcient (Tau) 2 equally to this work. R2 Coefcient of determination (R ) Electronic supplementary material The online version of this QM Quantum mechanics article (https ://doi.org/10.1007/s1082 2-020-00295 -0) contains MM Molecular mechanics supplementary material, which is available to authorized users. * Mehtap Işık [email protected] Extended author information available on the last page of the article Vol.:(0123456789)1 3 336 Journal of Computer-Aided Molecular Design (2020) 34:335–370 Introduction The log P challenge examines how well we model transfer free energy of molecules between diferent solvent environ- The development of computational biomolecular modeling ments in the absence of any complications coming from methodologies is motivated by the goal of enabling quanti- predicting protonation states and p Ka values. Assessing tative molecular design, prediction of properties and biomo- log P prediction accuracy also allows evaluating methods lecular interactions and achieving a detailed understanding for modeling protein-ligand afnities in terms of how well of mechanisms (chemical and biological) via computational they capture solvation efects. predictions. While many approaches are available for making such predictions, methods often sufer from poor or unpredict- History and motivation of the SAMPL challenges able performance, ultimately limiting their predictive power. It is often difcult to know which method would give the most The SAMPL blind challenges aim to focus the feld of quan- accurate predictions for a target system without extensive titative biomolecular modeling on major issues that limit the evaluation of methods. However, such extensive compara- accuracy of protein-ligand binding prediction. Companion tive evaluations are infrequent and difcult to perform, partly exercises such as the Drug Design Data Resource (D3R) because no single group has expertise in or access to all rel- blind challenges aim to assess the current accuracy of bio- evant methods and also because of the scarcity of blind experi- molecular modeling methods in predicting bound ligand mental data sets that would allow prospective evaluations. In poses and afnities on real drug discovery project data. D3R addition, many publications which report method comparisons blind challenges serve as an accurate barometer for accuracy. for a target system constructs these studies with the intention However, due to the confation of multiple accuracy-limit- of highlighting the success of a method being developed. ing problems in these complex test systems, it is difcult The Statistical Assessment of the Modeling of Proteins and to derive clear insights into how to make further progress Ligands (SAMPL) Challenges [http://sampl chall enges .githu towards better accuracy. b.io] provide a forum to test and compare methods with the Instead, SAMPL seeks to isolate and focus attention on following goals: individual accuracy-limiting issues. We aim to feld blind challenges just at the limit of tractability in order to identify 1. Determine prospective predictive power rather than underlying sources of error and help overcome these chal- accuracy in retrospective tests. lenges. Working on similar model systems or the same target 2. Allow a head to head comparison of a wide variety of with new blinded datasets in multiple iterations of prediction methods on the same data. challenges maximize our ability to learn from successes and failures. Often, these challenges focus on physical properties Regular SAMPL challenges focus attention on modeling of high relevance to drug discovery in their own right, such areas that need improvement, and sometimes revisit key test as partition or distribution coefcients critical to the devel- systems, providing a crowdsourcing mechanism to drive pro- opment of potent, selective, and bioavailable compounds gress. Systems are carefully selected to create challenges of (Fig. 1). gradually increasing complexity spanning between predic- tion objectives that are tractable and that are understood to be slightly beyond the capabilities of contemporary methods. So SAMPL5 logD Challenge far, most frequent SAMPL challenges have been on solvation and binding systems. Iterated blind prediction challenges have + played a key role in driving innovations in the prediction of cyclohexane water physical properties and binding. Here we report on a SAMPL6 SAMPL6 pKa Challenge SAMPL6 logP Challenge log P Challenge on octanol-water partition coefcients, treat- + ing molecules resembling fragments of kinase inhibitors. This K is a follow-on to the earlier SAMPL6 p a Challenge which octanol included the same compounds. water water + The partition coefcient describes the equilibrium concen- tration ratio of the neutral state of a substance between two phases: Fig. 1 The desire to deconvolute the distinct sources of error contrib- [unionized solute]octanol uting to the large errors observed in the SAMPL5 log D Challenge log P = log10 Kow = log10 K [unionized solute] (1) motivated the separation of p a and log P challenges in SAMPL6. water K The SAMPL6 p a and log P Challenges aim to evaluate protonation state predictions of small molecules in water and transfer free energy predictions between two solvents, isolating these prediction problems 1 3 Journal of Computer-Aided Molecular Design (2020) 34:335–370 337 The partition coefficient (log P) and the distribution accuracy such as a heterogeneous environment with poten- coefcient (log D) are driven by the free energy of transfer tially micelle-like bubbles [3–6], resulting in relatively slow from an aqueous to a nonpolar phase. Transfer free energy solute transitions between environments [4, 7]. The precise of only neutral species is considered for log P, whereas both water content of wet octanol is unknown, as it is afected by neutral and ionized species contribute to log D. Such sol- environmental conditions such as temperature as well as the ute partitioning models are a simple proxy for the transfer presence of solutes, the organic molecule of interest, and salts free energy of a drug-like molecule to a relatively hydro- (added to control pH and ionic strength). Inverse micelles tran- phobic receptor binding pocket, in the absence of specifc siently formed in wet octanol create spatial heterogeneity and interactions. Protein-ligand