Full Papers Chemistry—Methods doi.org/10.1002/cmtd.202000044 1 2 3 A Machine Learning-Enabled Autonomous Flow Chemistry 4 5 Platform for Process Optimization of Multiple Reaction 6 Metrics 7 8 Mohammed I. Jeraal,[b] Simon Sung,[b] and Alexei A. Lapkin*[a, b] 9 10 11 Self-optimization of chemical reactions using machine learning reaction conditions and trade-offs (Pareto fronts) between 12 multi-objective algorithms has the potential to significantly conflicting optimization objectives, such as yield, cost, space- 13 shorten overall process development time, providing users with time yield, and E-factor, in a data efficient manner. Advanta- 14 valuable information about economic and environmental geously, the robust system consists of exclusively commercially 15 factors. Using the Thompson Sampling Efficient Multi-Objective available equipment and a user-friendly MATLAB graphical user 16 (TS-EMO) algorithm, the self-optimization flow chemistry system interface, and was shown to autonomously run 131 experiments 17 in this report demonstrates the ability to identify optimum over 69 hours uninterrupted. 18 19 20 1. Introduction 21 22 Despite the prevalence of established techniques such as 23 Design of Experiments (DoE), reaction optimization is still often 24 a difficult and time-consuming task for chemists.[1] Identifying 25 Figure 1. General flow chart of a reaction self-optimization system. where improvements can be made is challenging due to the 26 large number of process variables with many different possible 27 combinations that should be tested. This issue can be alleviated 28 using self-optimizing systems that combine programmable self-optimizing systems utilize single-objective optimization 29 chemical handlers, a machine-learning reaction optimization algorithms, such as the Nelder-Mead simplex (NMSIM) and 30 algorithm, and online analytical techniques in a real-time Stable Noisy Optimization by Branch and FIT (SNOBFIT).[3–5] 31 adaptive feedback optimization loop (Figure 1). Examples of Owing to the significantly increased complexity of multiple 32 analytical methods suitable for self-optimising experimental objective optimization, there are few algorithms that have been 33 systems include gas chromatography (GC), high-performance demonstrated to efficiently perform this task. Whilst multiple 34 liquid chromatography (HPLC), mass spectrometry (MS), in-situ objectives can be scalarized into a single function, the 35 infrared spectroscopy (IR) and nuclear magnetic resonance weighting given to individual objectives is subjective when 36 (NMR) spectroscopy.[2] A significant advantage of these types of compared to multi-objective optimization. 37 systems is that the optimization procedure can be entirely Another key point for multi-objective algorithms is that 38 automated, where no user intervention is required. objectives sometimes compete with one another (e.g. yield vs. 39 Reaction optimization conducted by chemists is typically cost), which makes it is impossible to find a single set of 40 measured against multiple performance criteria such as yield, ‘utopian’ conditions that correspond with optimal values for 41 cost, impurities profile, and environmental impacts. Therefore, both objectives. One representation of competing multi- 42 the ability for the automated process to self-optimize for objective optimization is a Pareto front (Figure 2),[6] which is a 43 multiple objectives is highly desirable. The majority of existing set of non-dominated data points where either objective cannot 44 be improved without having a detrimental effect on the other, 45 i.e. showing the trade-off between objectives. An example of 46 [a] Prof. A. A. Lapkin Department of Chemical Engineering and Biotechnology an algorithm for efficient multi-objective reaction optimization 47 University of Cambridge is the open-source Thompson Sampling Efficient Multi-Objec- 48 Cambridge CB3 0AS (UK) tive (TS-EMO).[7] Lapkin and co-workers[6,8–10] have demonstrated 49 E-mail: [email protected] [b] Dr. M. I. Jeraal, Dr. S. Sung, Prof. A. A. Lapkin the quality of the generated Pareto fronts, as well as the 50 Cambridge Centre for Advanced Research and Education in Singapore Ltd. algorithm’s efficiency at identifying them, when compared with 51 1 Create Way, CREATE Tower #05-05 alternative algorithms such as ParEGO.[11] Alternative examples 52 138602, Singapore multi-objective algorithms[12] developed for chemical process 53 Supporting information for this article is available on the WWW under https://doi.org/10.1002/cmtd.202000044 include Phoenics[13] and Chimera.[14] 54 © 2020 The Authors. Published by Wiley-VCH GmbH. This is an open access The application of flow chemistry over batch methods for 55 article under the terms of the Creative Commons Attribution License, which self-optimizing systems has significant advantages. As well as 56 permits use, distribution and reproduction in any medium, provided the original work is properly cited. being inherently safer under high temperature and pressure 57 Chemistry—Methods 2021, 1, 71–77 71 © 2020 The Authors. Published by Wiley-VCH GmbH Wiley VCH Donnerstag, 14.01.2021 2101 / 188507 [S. 71/77] 1 Full Papers Chemistry—Methods doi.org/10.1002/cmtd.202000044 conditions. Here we also aim to further investigate the 1 optimization behavior of the TS-EMO Bayesian optimizer with 2 respect to exploitation vs exploration of experimental parame- 3 ter space. 4 5 6 2. Results and Discussion 7 8 The case study reaction was the aldol condensation reaction 9 between benzaldehyde (1) and acetone (2), catalyzed by 10 sodium hydroxide (3) base, to give the desired benzylideneace- 11 tone (4) product (Scheme 1). The possible side-reactions to 12 form dibenzylideneacetone (5) or acetone polymerization side- 13 products represent an ideal challenge for careful control of 14 reaction conditions chosen by the algorithm. 15 The self-optimization system utilized in this work features 16 exclusively commercially available equipment and the TS-EMO 17 Figure 2. An illustration of a Pareto front (made up of non-dominated multi-objective optimization algorithm (Figure 3). The flow 18 solutions) in a system with two competing optimization objectives, where chemistry equipment consists of two Vapourtec R2 modules 19 values in the infeasible region under the Pareto front are inaccessible to the optimization process. and a R4 reactor module for controlling solution flows and 20 reactor temperatures respectively. These parameters are con- 21 trolled from within the software provided by the manufacturer. 22 conditions (process intensification conditions), in situ analysis Designed for mesoscale flow chemistry,[22] the system uses 23 and closed-loop optimization systems are easier to implement plug-flow modelling by calculating the flow rates and pump 24 in flow conditions as automated direct reaction sampling of the timings in relation to the desired reaction-zone plug sizes, 25 reaction solution can be performed using in-line small volume determination of solution compositions within a plug, and 26 injectors or using non-invasive spectroscopic sampling. Further- automated signaling to reaction samplers and analytical equip- 27 more, subsequent flow chemistry reactions can be conveniently ment when the system is deemed to have reached steady-state. 28 initiated with different continuous reaction variables by modu- These features allow for easy implementation of direct reaction 29 lating reactor temperatures and flow rates. Conversely, the mixture sampling at steady-state using a microliter injector into 30 screening of continuous variables in batch reactions is an online HPLC-UV instrument. A bespoke MATLAB user inter- 31 inefficient, typically requiring expensive robotic equipment.[15] face was developed to control all aspects of the self- 32 Self-optimization flow systems reported in the literature optimization process, including control of physical equipment 33 typically utilize custom-designed setups (consisting of pumps, through interface with commercial software, creation of training 34 reactors, samplers, and analytical equipment) interfaced with in- data sets, reading HPLC data and calculation of optimization 35 house software, which could be detrimental to the widespread objectives, and the complete, autonomous execution of flow 36 adoption and rapid development of these tools. Furthermore, chemistry experiments.[23] This process was repeated iteratively 37 systems are sometimes developed for specific reactions, where until the user terminated the MATLAB environment. It should 38 modifying a system for a different reaction often requires be noted that any downstream processes, such as purification 39 considerable effort and time, even by experts.[16] In contrast, the steps, were not taken into consideration in this work. Therefore, 40 applications of commercially available modular flow chemistry 41 systems, for example by Vapourtec, have been demonstrated to 42 be effective in conducting many different reactions.[17–21] 43 Furthermore, for more complex and scripted applications such 44 as self-optimization, some systems can be remotely controlled 45 through their standard software packages using application 46 programming interfaces (API) written by manufacturers from 47 popular programming environments in languages such as 48 MATLAB or Python. 49 In this study we aim to further develop autonomous self- 50 optimization flow chemistry systems, by developing a robust 51 implementation, based on commercially available equipment 52