2015 IEEE International Conference on Big Data (Big Data) A Neural Network Meta-Model and its Application for Manufacturing David Lechevalier Steven Hudak Le2i, University of Maryland, Université de Bourgogne Baltimore County, 21250, MD, US BP 47870, 21078 Dijon, France [email protected] [email protected] Sebti Foufou Ronay Ak, Y.Tina Lee CSE Department, College of Engineering, Systems Integration Division, Engineering Laboratory Qatar University, Qatar National Institute of Standards and Technology [email protected] Gaithersburg, MD 20899, USA [email protected] [email protected] Abstract—Manufacturing generates a vast amount of data supply chain level [9], [10]. These applications demonstrate how both from operations and simulation. Extracting appropriate manufacturers can make their systems smarter using neural information from this data can provide insights to increase a networks. However, developing an NN model requires data manufacturer’s competitive advantage through improved science knowledge that manufacturers often do not have. As a sustainability, productivity, and flexibility of their operations. contribution to automating the application of neural networks in Manufacturers, as well as other industries, have successfully manufacturing, we propose an NN meta-model (NNMM) that applied a promising statistical learning technique, called neural encapsulates data scientist knowledge about neural networks. networks (NNs), to extract meaningful information from large We introduce an algorithm to automatically manipulate a neural data sets, so called big data. However, the application of NN to network model (NNM), built using the meta-model and manufacturing problems remains limited because it involves the illustrate how manufacturers can leverage the meta-model to specialized skills of a data scientist. This paper introduces an approach to automate the application apply neural networks to their manufacturing systems. of analytical models to manufacturing problems. We present an The paper is organized as follows. Section II provides the NN meta-model (MM), which defines a set of concepts, rules, and use case scenario and required background about both neural constraints to represent NNs. An NN model can be automatically networks and meta-models. Section III presents the NNMM and generated and manipulated based on the specifications of the NN an algorithm to manipulate an NNM created using this meta- MM. In addition, we present an algorithm to generate a predictive model. Section IV introduces a manufacturing use case to model from an NN and available data. The predictive model is illustrate the capabilities of using the meta-model in the represented in either Predictive Model Markup Language (PMML) or Portable Format for Analytics (PFA). Then we manufacturing area. We conclude this paper by presenting future illustrate the approach in the context of a specific manufacturing work that supports automation of the neural networks system. Finally, we identify future steps planned towards later application on a manufacturing system. This paper will be of implementation of the proposed approach. interest to manufacturers that look to apply neural networks on their manufacturing systems, and schema developers who look Keywords—neural network; meta-model; data analytics; to represent data science knowledge. PMML; manufacturing; II. CONTEXT AND BACKGROUND I. INTRODUCTION In this section, we introduce the use case scenario and we The manufacturing industry generates a large amount of data provide background about both neural networks and meta- [1]. At each manufacturing level (i.e., from the shop floor to the models. enterprise level), data is produced and recorded to monitor the A. Use case scenario operations that occur. One way to extract useful information from “big data” is through the application of data analytics. This section describes a case study that focuses on predicting Applying data analytics on manufacturing data is a promising the energy consumed by a milling machine tool. The goal of this way to improve the efficiency of the manufacturing system as case study is to predict estimated energy consumption well as reducing the cost of production at every manufacturing corresponding to the machine’s operational parameters. The level [2]. In particular, neural networks (NN), a statistical following three input variables are used to estimate the required technique, has been widely used in data analytics. Examples of energy to manufacture a certain workpiece: the NN applications are available at the process level [3], [4], at Feed rate: the velocity at which the tool is fed the machine level [5], [6], at the factory level [7], [8] and at the 978-1-4799-9926-2/15/$31.00 ©2015 IEEE 1428 Spindle speed: rotational speed of the tool the variables when these relationships cannot be captured in a regression model. At both the input and hidden layers, there is a Depth of cut: the actual depth of material that the tool is bias neuron. The bias neuron, which allows shifting the removing activation function to the left or right for improving the learning A milling machine tool obviously involves more parameters. process, is a neuron with a constant output. This neuron is For simplicity, we decided to only keep these three parameters treated as a regular neuron in the associated layer. Finally, the which have the biggest impact on the energy consumption in the output layer contains the output neuron that represents the milling process. predicted energy in our example. The free parameters w called weights are assigned to the edges that connect the different B. Regression Analysis and Neural Network (NN) Neuron We present the characteristics of regression analysis and Weighted (or node) neural networks. Regression analysis is a statistical process used edge Sigmoid Output to investigate the relationships between variables [11]. function Regression techniques can be roughly divided into “linear” and “non-linear” regressions. A dataset is defined as a 풟 = feed rate {(푥푖1, 푥푖2, … , 푥푖푝, 푦푖)| = 1, … , 푛}, where p and n indicate the number of input variables and the number of samples in our spindle dataset, respectively. 푦 is the corresponding output target scalar 푖 Energy for each input vector 풙푖 [12]. speed NNs are ooriginally inspired by the function of the neurons in the brain. They are typically used for classification and depth forecasting. They are frequently used as an alternative to standard nonlinear regression and cluster analysis techniques. of cut NNs are composed of computing units (called neurons or nodes) operating in parallel. These units are arranged in different layers bias and interconnected by weighed edges (called synapses). A layer is the term used for a vertical row of neurons. Each of these computing units performs a few simple operations and Input Hidden Output communicates the results to its neighboring units. From a Layer Layer Layer mathematical viewpoint, NNs consist of a set of nonlinear (e.g., sigmoidal) basis functions with free parameters; i.e., w or Fig. 1. Example of a MLP NN for estimating a quantity of interest weights, that are adjusted. The objective of the adjustment is to layers. During the training, these weights are adjusted to minimize the error associated with regression in an iterative minimize the error between the output value of the NN and the process called training using a dataset. The basis functions are real output value for a given data sample. In essence the weights called activation functions. represent the sensitivity of the output to the input variable. There are many types of NNs. Feed-forward neural networks Equation (1) introduces how to compute the output value using (FNNs) and recurrent neural networks (RNNs) are the two main the weights. types. A RNN has neurons that transport a signal back through Each layer receives input signals generated by the previous the network (at least one feedback connection in recurrent layer, produces output signals through an activation function network) whereas FNNs feed outputs from individual neurons (e.g. a sigmoid function) and distributes them to the subsequent forward to one or more neurons or layers in the network [13], layer through the neurons. The network output is given by the [14]. Multilayer perceptron (MLP) neural networks and Radial following expression: basis function (RBF) neural networks are two of the most common types of FNNs used as empirical nonlinear regression ℎ 푝 models. RBF networks use a radial basis function, i.e., a 푓(푿; 풘) = 푦̂(푿) = 휓(푤0푙푏 + ∑푗=1 푤푗푙휙(푤0푗푏 + ∑푘=1 푤푖푗풙푘)) , (1) Gaussian kernel, as the activation function. RBFs networks have similar universal approximation capabilities as MLP networks. where 푿 is an input vector with p entries, 푿 = (풙1, 풙2, … , 풙푝), p th For the theory and application of the RBF networks, we refer the is the number of input signals (variables), 풙푘 is the k input readers to [13], [15]. signal, w is the weight vector, ℎ indicates the number of hidden neurons, 푤 is the synaptic weight from ith neuron to jth A feed-forward neural network that represents our use case 푖푗 is illustrated in Fig. 1. The neural network is composed of an neuron, 휙() and 휓() are the activation (transfer) functions from input layer, a hidden layer and an output layer. The input layer input layer to hidden, and hidden layer to output layer, is the layer containing the input neurons. In our example, the respectively, and b stands for the bias factor. input neurons represent the three input parameters: feed rate, An estimate 풘̂ of 풘 can be obtained by a training procedure spindle speed and depth of cut. The hidden layer contains the aimed at minimizing the quadratic error function, E, on a hidden neurons. The hidden layers are used to represent the training set: relationships between the input layer and the output layer. The hidden layers allow a representation of the relationships among 1429 푛 2 domain-specific abstractions representing all of the relevant 퐸(푤) = ∑푖=1(푦̂푖 − 푦푖) , (2) domain-specific core components.