21st International Workshop on Principles of Diagnosis, 2010 Event-driven Data Mining Techniques for Automotive Fault Diagnosis Chaitanya Sankavaram1, Anuradha Kodali1, Diego Fernando Martinez Ayala1, Krishna Pattipati1, Satnam Singh2, and Pulak Bandyopadhyay2 1 Electrical and Computer Engineering, University of Connecticut, Storrs, CT 06269 USA E-mail: [email protected] 2 Diagnosis & Prognosis Group, India Science Lab, General Motors Global Research and Development, GM Technical Centre India Pvt Ltd, Bangalore, INDIA E-mail: [email protected] ABSTRACT Automotive OEMs collect a variety of on-board vehicle health data via telematics and off-board data via dealer The increasing sophistication of electronics in diagnostics services. These data sources acquire vehicular systems is providing the necessary different types of vehicle data at different sampling information to perform data-driven rates. For example, dealer diagnostic data is collected diagnostics. Specifically, the advances in when a vehicle comes for repair at a dealer shop; the automobiles enable periodic acquisition of warranty data, collected infrequently, includes the data from telematics services and the diagnostic trouble codes (DTCs), freeze frame data associated dealer diagnostic data from (engineering variables/PIDs), repairs/replacement vehicles; this requires a data-driven actions, and structured/unstructured text in the form of framework that can detect component customer verbatim. The fleet data is collected at a degradations and isolate the root causes of much higher sampling frequency (e.g., every few failures. The event-driven data consists of ignition cycles) for overall health of vehicle diagnostic trouble codes (DTCs) and the subsystems, such as the engine and/or transmission concomitant parameter identifiers (PIDs) system, emission system, airbag system, anti-lock brake collected from various sensors, customer system, tire pressure; this data is gathered even when complaints (CCs), and labor codes (LCs) the vehicle is functioning normally. However, what is associated with the repair. In this paper, we needed is an early warning capability that continuously discuss a systematic data-driven diagnostic monitors the data, detects, isolates and estimates the framework featuring data pre-processing, data severity of faults (viz., fault detection and diagnosis) visualization, clustering, classification, and based on models that includes cross-subsystem fault fusion techniques and apply it to field failure propagation effects, and relates the detected datasets. The results demonstrated that the degradations in vehicles to accurate remaining life-time support vector machine (SVM) classifier with predictions (viz., prognosis) of replaceable DTCs and customer complaints as features components. provides the best accuracy (74.3%) compared Methods for fault diagnosis can be classified as to any other classifier and that a tree- being associated with one or more of the following structured classifier with SVM as the base three approaches: model-based, knowledge-based, or classifier at each node achieves approximately data-driven. What if a mathematical model (for model- 75.2% diagnostic accuracy. based diagnosis) or cause-effect graph model of system failures and their manifestations (for knowledge-based approach to diagnosis) is not available? The Data- 1 INTRODUCTION driven approach to fault diagnosis is an alternative, The relentless competition among automotive OEMs, provided that system monitoring data is available. A increased demands from customers for dynamically- data-driven approach to fault diagnosis has close controlled safety systems and growing dependence on relationship with pattern recognition, wherein one electronics are creating the need for a continuous learns classification rules directly from the data, rather monitoring system that tracks and identifies the trends than using mathematical models or a knowledge-based and sources of component degradations prior to failure. approach. Due to its simplicity and adaptability, 1 21st International Workshop on Principles of Diagnosis, 2010 Fig. 1. Diagnostic Process Overview customization of a data-driven approach does not viz., its inability to diagnose anticipated, but require an in-depth knowledge of the system. However, unobserved faults. We recommend that a recursive data-driven techniques provide no information on classifier should be developed that can incrementally unobserved faults, even though they may be anticipated adapt with the observed cases for unanticipated faults. (Sankavaram et. al., 2009). The proposed diagnostic process is generic; it can be In this paper, we propose a systematic data-driven applied to a wide range of subsystems across vehicle diagnostic framework for fault diagnosis. The key types and models. We demonstrate the process on two features of the proposed framework include data pre- datasets (dataset 1 and dataset 2) gleaned from field processing, visualization, clustering, classification, and failure databases. fusion algorithms to detect and isolate faults and to The paper is organized as follows: In Section 2, we reduce no-trouble found rates and warranty costs. describe our data-driven diagnostic framework. In Often, the data acquired via onboard diagnostic systems Section 3, we describe the characteristics of the (telematics), dealer diagnostics etc., require data pre- datasets, demonstrate our proposed framework and processing techniques in order to process the DTCs, discuss the classification results. Finally, the paper engineering variables and the vehicle information. concludes with a summary in Section 4. Hence, the data-preprocessing technique plays a crucial role in fault diagnosis. Furthermore, in order to 2 DATA-DRIVEN DIAGNOSTIC diagnose the faults of interest, a number of classifier FRAMEWORK techniques are employed, viz., support vector machines (SVM), probabilistic neural network (PNN), Gaussian Our data-driven diagnostic framework, shown in Fig. 1, mixture models (GMM), k-nearest neighbor (KNN) consists of visualization, feature selection, clustering, classifier and so on (Bishop, 2006; Duda et al., 2001). classification, and fusion techniques. Each process is The diagnostic process also exploits clustering explained in detail in the following subsections. techniques, such as Principal Component Analysis (PCA), Partial Least Squares (PLS) and Linear Vector 2.1 Graphical Visualization of Data Quantization (LVQ) especially to cluster/ group the ambiguous faults in order to improve the classification Data visualization helps in assessing the level of accuracy. Furthermore, by examining the confusion difficulty of classification problem and also helps in matrices, we suggest a tree-structured classifier, where selecting data pre-processing techniques for better the ambiguous labor codes (LCs) are grouped together diagnosis (e.g., grouping or subdividing labor codes, and a classifier is applied on the subset of LCs. We also etc.). We employed a variety of data visualization illustrate a major limitation of data-driven approaches, techniques viz., Self Organizing Map (SOM), PCA, 2 21st International Workshop on Principles of Diagnosis, 2010 Fig. 2. Histograms of Top PID, DTCs and Customer Complaints PLS, histograms, etc., as part of our diagnostic process. n( O , LC ) 1 Pˆ(/) O LC ji (2) For instance, Fig. 2 shows the histograms of the top jin( LC ) | LC | PID and DTCs conditioned on the LCs. We could i observe that the DTCs (also CCs) may provide better where |LC| is the number of labor codes. The isolation of labor codes when compared to the top PID dependency matrix can be either binary (0 or 1 (hard)) alone (according to information gain criterion) because or probabilistic (soft). These fault models provide an there is no significant overlap of labor codes with alternative method for inferring the LCs from the DTCs and CCs. This is borne out by our classification classified DTCs or CCs (see Fig1). analysis later. 2.3 Feature Selection 2.2 Bi-partite Fault models The main idea of feature selection is to choose a subset of PIDs by eliminating those with little or no predictive Fault model is a probabilistic dependency matrix which information. We rank ordered PIDs via mutual depicts the cause-effect relationships between the information gain algorithm and selected the minimum failure modes (LCs) and the tests (DTCs, CCs). We number of PIDs required for classification based on the have derived the fault models between (i) LCs and diagnostic accuracy on test case data. The mutual DTCs, (ii) LCs and CCs, and (iii) LCs and DTC-CC information (MI) (Cover et. al., 1991) between a combinations via maximum likelihood estimation of feature, F and a class, C is given by, probabilities given by, p( F | C ) MIFC(,) PCpFC ()[(|)log dF ] (3) nij n(,) O j LC i pF() pˆ Pˆ(/) O LC (1) C F ijn j i n() LC iiHistograms of relevant probability density functions where Oj is DTCj or CCj or (DTC,CC)j, nij is the number from data are used to compute (3). We seek minimum of times Oj is associated with LCi and ni is the total number of PIDs to decrease the implementation number of observed cases with LCi. In order to avoid complexity of classification approaches. the problem with ML estimate i.e., the possibility of having a zero probability because of an unseen 2.4 Classification combination of (O LC ) in the training data, we use j, j Our framework features a number of statistical Laplacian smoothing (Metzler et al., 2004) given by, 3 21st International Workshop