Deep Variational Autoencoder: An Efficient Tool for PHM Frameworks Ryad Zemouri, Melanie Levesque, Normand Amyot, Claude Hudon, Olivier Kokoko To cite this version: Ryad Zemouri, Melanie Levesque, Normand Amyot, Claude Hudon, Olivier Kokoko. Deep Varia- tional Autoencoder: An Efficient Tool for PHM Frameworks. 2020 Prognostics and Health Man- agement Conference (PHM-Besançon), May 2020, Besancon, France. pp.235-240, 10.1109/PHM- Besancon49106.2020.00046. hal-02868384 HAL Id: hal-02868384 https://hal-cnam.archives-ouvertes.fr/hal-02868384 Submitted on 7 Jul 2020 HAL is a multi-disciplinary open access L’archive ouverte pluridisciplinaire HAL, est archive for the deposit and dissemination of sci- destinée au dépôt et à la diffusion de documents entific research documents, whether they are pub- scientifiques de niveau recherche, publiés ou non, lished or not. The documents may come from émanant des établissements d’enseignement et de teaching and research institutions in France or recherche français ou étrangers, des laboratoires abroad, or from public or private research centers. publics ou privés. Deep Variational Autoencoder: an efficient tool for PHM frameworks. Ryad Zemouri∗,Melanie´ Levesque´ y, Normand Amyoty, Claude Hudony and Olivier Kokokoy ∗Cedric-Lab, CNAM, HESAM universite,´ Paris (France), [email protected] y Institut de Recherche d’Hydro-Quebec´ (IREQ), Varennes, QC, J3X1S1 Canada Abstract—Deep learning (DL) has been recently used in several Variational Autoencoder applications of machine health monitoring systems. Unfortu- nately, most of these DL models are considered as black-boxes with low interpretability. In this research, we propose an original PHM framework based on visual data analysis. The most suitable space dimension for the data visualization is the 2D-space, Encoder Decoder which necessarily involves a significant reduction from a high- dimensional to a low-dimensional data space. To perform the data analysis and the diagnostic interpretation in a PHM framework, Latent space a Variational Autoencoder (VAE) is used jointly with a classifier. The proposed model was evaluated to automatically recognize individual Partial Discharge (PD) sources for hydro generators monitoring. Index Terms—Variational Autoencoder, data visualization, sys- Fig. 1. Schematic architecture of a variational deep autoencoder. tem health management, diagnosis analysis. I. INTRODUCTION Encoder Deep Neural Networks (DNN) and Deep learning (DL) Latent space represent, nowadays, the most effective machine learning tech- 2D Data Visualisation nology in recent applications of Machine Health Monitoring Systems (MHMS) and fault diagnosis [1], [2], [3], [4], [5]. The Fig. 2. Data visualization by dimension reduction. quality of the training data is an important factor that affects the performance of these DL architectures. In spite of their superior discrimination power in many fields, the DNN often II. DEEP VARIATIONAL AUTOENCODER FOR DATA lacks interpretability. Usually, it is very hard to understand why VISUALIZATION the classifier makes a given decision, and in what situations For the data visualization, the most suitable space dimension it is reliable [6]. The interpretability of a classifier for an for a human brain is the 2D-space. This necessarily involves efficient diagnosis is especially important in machine health data dimension reduction from a high-dimensional to a low- monitoring systems. Therefore, most of the DNN classification dimensional space. Classical methods from statistics such as models are considered as black-boxes. The interpretability principal components analysis (PCA) [9] and t-SNE [10] has of their internal processing mechanism through the hidden been an important topic in visual analytics. More recently, the layers is usually challenging. Indeed, the high-dimensional deep variational autoencoders (VAE) have been successfully space data transformation goes beyond the capacity of human used for feature dimension reduction [8], [11], [5]. interpretation [7]. A VAE is a special extension of the autoencoder (AE) which In this paper, a new PHM framework based on visual data is an unsupervised Neural Network (NN) trained to reproduce analysis and interpretation is presented (Fig 3). More reliable the input vector x [12], [8]. The VAE is composed by two interpretations and analysis of classifier performance can be separate multilayered NNs: an encoder and a decoder as illus- made by visualizing the 2D latent space of a VAE. This trated in Fig. 1, parameterized by φ and θ, respectively. The work is the progression of the results previously published [8] first NN encodes the input data x into a latent representation where the reader can find additional explanations. The paper z by the encoder function z = fφ(x), whereas the second one is organized in three main sections. First, some theoretical decodes this latent representation onto x^ = hθ(z) which is a backgrounds of the VAE are given. Then, the deep learning reconstruction of the original data. In a VAE, an equal number PHM framework is presented. Finally, some practical results of units are used in the input/output layers while less units are obtained on hydro generators monitoring are detailed. used in the latent space. Hydrogenerator 7. Maintenance decision Stator making 1. Data aquisition and pre-processing PD activity measured using the 2D PDA technique Data Visualisation, analysis and PD rate (PD/s) Amplitude (mV) Fault Class N°1 interpretation Normal Class z1(t0) RUL pdf 2. Input vector Assessing the extent of a Degradation1 z1(t3) z1(t1) Failure threshold z1(t2) definition degradation z2(t2) z2(t1) Conflict zones z2(t0) 25 40 Degradation2 RUL pdf z2(t3) A B Failure threshold 20 30 15 20 Fault Class N°2 10 5 10 0 0 -5 -10 -10 hand designed -15 -20 -20 features -30 -25 Expert Rules -30 -40 -25 -20 -15 -10 -5 0 5 10 15 20 25 -40 -30 -20 -10 0 10 20 30 40 Diagnosis 6 4 2 6. Prognosis 0 -2 -4 3. Labeling step -6 -8 -10 -8 -6 -4 -2 0 2 4 6 Building of the learning Rejection of the data set outliers 5. Model Training 4. Identification of the outliers Fig. 3. The PHM framework used by Hydro-Quebec for hydro generator diagnosis and prognosis. 25 40 VAE becomes a popular generative model by combining A B 20 30 Bayesian inference and the efficiency of the NNs to obtain 15 20 a nonlinear low-dimensional latent space [8]. The Bayesian 10 5 10 inference is obtained by an additional layer used for sampling 0 0 the latent vector z with a prior specified distribution p(z), -5 usually assumed to be a standard Gaussian N (0; I), where -10 -10 -15 -20 I is the identity matrix. Each element z of the latent layer -20 i -30 is obtained by z = µ + σ ., where µ and σ are the ith -25 i i i i i -30 -40 components of the mean and standard deviation vectors, is -25 -20 -15 -10 -5 0 5 10 15 20 25 -40 -30 -20 -10 0 10 20 30 40 a random variable following a standard Normal distribution Fig. 4. The latent space obtained by a variational autoencoder [8]: A. deep ( ∼ N (0; 1)). Unlike the AE which generates the latent vector learning end-to-end feature design without expert knowledge, B. Conventional hand-crafted feature design with expert knowledge. z, the VAE generates vector of means µi and standard devi- ations σi. This a major advantage that gives more continuity in the latent space than the original AE. model including the feature extraction and the classification When the VAE is trained, each function (i.e., the encoder module are usually trained jointly. Conversely, the conven- and the decoder) can be used separately, either to reduce the tional hand-crafted feature design requires a significant amount space dimension by encoding the input data, or to generate of expertise from the practitioner, especially for complex synthetic samples by decoding new variables from the latent systems, but needs less labeled data. We have seen in our space as seen in the 2D Data visualization in Fig. 2. previous study [8] that the more we put knowledge in the input III. DEEP LEARNING PHM FRAMEWORK vector, the more the data space becomes discriminant, thus making the classification easier. Separation between classes Figure 3 shows the PHM framework used by Hydro- is transformed from non-linear separability with high overlap- Quebec for hydro generator diagnosis and prognosis. This ping clusters, as illustrated in Fig. 4.A, into a pseudo-linear one PHM framework is based on a main central function which is with less overlapping clusters as seen in Fig. 4.B. This second the visual data analysis and interpretation. Seven main steps representation based on the conventional hand-crafted design are considered: 1. Data acquisition and pre-processing, 2. Input with expert knowledge reduces the ambiguities and conflict vector definition, 3. Labelling step, 4. Outliers identification, areas between classes. 5. Diagnosis model training, 6. Assessing the extent of a degradation, 7. Maintenance decision making. Some of these B. Labeling the training dataset functions are detailed below. One of the main problems facing all industries is the massive A. Hand-designed features definition high dimensionality unlabeled data. To perform an intelligent The features definition in the machine health monitoring classification of such amount of data, experts must first label systems are categorized into conventional hand-crafted feature several measurement files for the model training process. To do design against deep learning end-to-end feature design. The this, experts faced two challenging problems: how to select the deep learning end-to-end structure enables the construction of most significant data for labeling? and what is the minimum the MHMS framework with less expert knowledge [1], but data size required to complete the training process that would needs to have a significant amount of labeled data. The whole be sufficient to define each class? One efficient way to help the PHM expert for selecting the Positive Class most suitable instances for the labeling step is to visualize Outliers and analyze the spatial distribution of the available dataset.