Architectures for Combining Discrete-Event Simulation and Machine Learning

Architectures for Combining Discrete-event Simulation and Machine Learning Andrew Greasley a Operations and Information Management Department, Aston University, Aston Triangle, Birmingham, U.K. Keywords: Discrete-Event Simulation, Software Architectures, Machine Learning, Reinforcement Learning. Abstract: A significant barrier to the combined use of simulation and machine learning (ML) is that practitioners in each area have differing backgrounds and use different tools. From a review of the literature this study presents five options for software architectures that combine simulation and machine learning. These architectures employ configurations of both simulation software and machine learning software and thus require skillsets in both areas. In order to further facilitate the combined use of these approaches this article presents a sixth option for a software architecture that uses a commercial off-the-shelf (COTS) DES software to implement both the simulation and machine learning algorithms. A study is presented of this approach that incorporates the use of a type of ML termed reinforcement learning (RL) which in this example determines an approximate best route for a robot in a factory moving from one physical location to another whilst avoiding fixed barriers. The study shows that the use of an object approach to modelling of the COTS DES Simio enables an ML capability to be embedded within the DES without the use of a programming language or specialist ML software. 1 INTRODUCTION To address these challenges this article investigates current options for combined DES and This article considers the combined use of simulation ML architectures and explores what these options can and machine learning (ML) which can be considered provide. In addition in order to remove the need for as two general approaches to computationally an interface with external ML software and to further predicting the behaviour of complex systems (Deist remove the need to code ML algorithms this article et al., 2019). A widely used simulation technique is presents a case study that demonstrates a software discrete-event simulation (DES) (Law, 2015). architecture of a DES that embeds an ML capability Robinson (2014) describes three options for implemented entirely within the COTS DES software developing DES of spreadsheets, programming Simio v11 (Smith et al., 2018) using the software’s languages and specialist simulation software standard process logic facilities. otherwise known as commercial off-the-shelf The article is organized as follows. The literature software (COTS). Hlupic (2000) reported that the review covers the combined use of COTS DES and majority (55.5%) of industrial users employ machine learning software and categorises them into simulators (COTS). However the number of six options for software architecture examples of the combined use of COTS DES and ML implementations. A further software architecture that is low and one reason for this may be due to the implements an ML algorithm within COTS DES is challenge of coding ML algorithms for DES presented. The study then outlines a use-case of this practitioners who may have little coding experience architecture with the integration of ML algorithms due to the common adoption of drag and drop within the COTS DES software Simio. The ML interfaces in COTS DES tools (Greasley and algorithms direct the movement of a robot, in the Edwards, 2019). Another challenge to the combined form of an automated guided vehicle (AGV), in a use of DES and ML put forward by Creighton and factory setting. The discussion section then evaluates Nahavandi (2002) is the need to provide an interface the current and presented architectures for combining between the ML agent and the (COTS) DES software. COTS DES and ML applications. a https://orcid.org/0000-0001-6413-3978 47 Greasley, A. Architectures for Combining Discrete-event Simulation and Machine Learning. DOI: 10.5220/0009767600470058 In Proceedings of the 10th International Conference on Simulation and Modeling Methodologies, Technologies and Applications (SIMULTECH 2020), pages 47-58 ISBN: 978-989-758-444-2 Copyright c 2020 by SCITEPRESS – Science and Technology Publications, Lda. All rights reserved SIMULTECH 2020 - 10th International Conference on Simulation and Modeling Methodologies, Technologies and Applications 2 LITERATURE REVIEW unsupervised learning categories, but as a technique that looks to maximise a reward signal instead of ML techniques can be classified into supervised trying to find hidden structure (Sutton and Barto, learning techniques that learn from a training set of 2018). labelled examples provided by a knowledgeable A literature review was undertaken to identify external supervisor and unsupervised learning which implementations of big data analytics applications is typically about finding structure hidden in such as ML in conjunction with COTS DES based on collections of unlabelled data. The main machine the criteria stated in Greasley and Edwards (2019). learning techniques are defined by Dasgupta (2018) This review is specific to COTS DES software and as association rules mining (ARM) which uses a machine learning applications. Machine learning rules-based approach to finding relationships applications are distinguished from data mining between variables in a dataset, decision trees (DT) examples in that machine learning uses algorithms generate rules that derive the likelihood of a certain that can learn from data and therefore can build outcome based on the likelihood of the preceding decision models that try to emulate regularities from outcome. In general, decision trees are typically training data in order to make predictions (Bishop, constructed similarly to a flowchart and belong to a 2006). The scope of the review means that a number class of algorithms that are often known as CART of articles that cover the combined use of simulation (Classification and Regression Trees). Support vector and ML are not included in this review. These articles machines (SVM) are used to classify data into one or either cover different types of simulation such as another category using a concept called hyperplanes, System Dynamics (Elbattah et. al, 2018) or non- artificial neural networks (ANN) are a network of COTS DES implementations such as DEVSimPy connected layers of (artificial) neurons which mimic (Capocchi et al., 2018), C (Chiu and Yih, 1995), neurons in the human brain that “fire” (produce an SimPy (Fairley et al., 2019), Psighos (Java) (Aguilar- output) when their stimulus (input) reaches a certain Chinea et al., 2019) and DESMO-J (Java) (Murphy et threshold and naïve Bayes classifier (NBC) employs al., 2019). Articles from the review that meet the a training set for classification. Reinforcement criteria of using a COTS DES are now categorised learning (RL) can be classified as a third paradigm of into 6 software architectures for employing COTS machine learning, not within the supervised and DES and ML with a further category to be presented in this article (Table 1). Table 1: Architectures for combining COTS DES and ML software. SOFTWARE ARCHITECTURE FOR DES COTS MACHINE INTERFACE REFERENCE COMBINING DES AND ML SOFTWARE LEARNING SOFTWARE 1. DES ‐> ML (OFFLINE) TECNOMATIX R DATA FILE Gyulai et al. (2014) ANYLOGIC KNIME DATA FILE Jain et al. (2017) ARENA SPSS DATA FILE Acqlan et al. (2017) 2. ML ‐> DES (OFFLINE) SIMPROCESS SPSS DATA FILE Glowacka et al. (2009) 3. DES ‐> ML ‐> DES TECNOMATIX MATLAB DATA FILE Bergmann et al. (2017) (OFFLINE) ANYLOGIC SCIKIT‐LEARN DATA FILE Cavalcante et al. (2019) WITNESS RAPIDMINER DATA FILE Priore et al. (2018) 4. DES ‐> ML (ONLINE) TECNOMATIX MATLAB WRAPPER Bergmann et al. (2015) 5. ML ‐> DES (ONLINE) ARENA SERVER Celik et al. (2010) 6. DES ‐> ML ‐> DES TECNOMATIX MATLAB WRAPPER Bergmann et al. (2014) (ONLINE) TECNOMATIX MATLAB WRAPPER Bergmann et al. (2017) QUEST MATLAB SERVER Creighton et al. (2002) 7. INTEGRATED SIMIO NONE NONE DES ‐> ML ML ‐> DES DES ‐> ML ‐> DES 48 Architectures for Combining Discrete-event Simulation and Machine Learning Figure 1 shows how the architectures are conditional and probability statements in the employed to enable synthetic data generated by a SimProcess COTS DES model. simulation to be used by an ML algorithm and the use Architecture 3 uses simulation to generate of an ML algorithm to provide decisions for a synthetic data that is used by ML algorithms which simulation. These two roles are combined in are subsequently employed in the simulation. If this architectures 3, 6 and 7 where simulation data is used option is chosen then a data file may be used for by an ML algorithm which is subsequently used to offline analysis such as in Bergmann et al. (2017) generate decisions for a simulation model. Each of the who outline the identification of job dispatching rules 7 architectures will now be described in more detail. built using a decision tree using the CART algorithm Architecture 1 uses synthetic data generated by a implemented in the MatLab toolbox. The decision simulation and held in a data file that is then used by tree is converted into decision rules which can then be ML algorithms. These algorithms are used stand- codified in the simulation software, in this case the alone and are not employed in the simulation model. Tecnomatix Plant scripting language Sim Talk. Gyulai et al. (2014) use the Tecnomatix Plant DES Cavalcante et al. (2019) use an Anylogic DES model software in conjunction with the random forest tree- to generate a database file which is subsequently used based machine learning technique using the R ML by the SciKit-Learn Python ML module. The results software. Jain et al. (2017) train ANN using the of the ML analysis are then saved as a file which Knime ML software from simulation output saved in serves as an input file for a simulation experiment. CSV files. Aqlan et al. (2017) use a traditional Priore et al. (2018) use simulation to generate training simulation methodology to develop a model of a and test sets which are used for a variety of machine high-end server fabrication process.

Load more