Wisdom of the Crowd for Fault Detection and Prognosis Yuantao Fan Supervisors: Thorsteinn Rögnvaldsson Sławomir Nowaczyk DOCTORAL THESIS | Halmstad University Dissertations no. 67 Wisdom of the Crowd for Fault Detection and Prognosis © Yuantao Fan Halmstad University Dissertations no. 67 ISBN 978-91-88749-42-0 (printed) ISBN 978-91-88749-43-7 (pdf) Publisher: Halmstad University Press, 2020 | www.hh.se/hup Printer: Media-Tryck, Lund Abstract Monitoring and maintaining the equipment to ensure its reliability and avail- ability is vital to industrial operations. With the rapid development and growth of interconnected devices, the Internet of Things promotes digitization of in- dustrial assets, to be sensed and controlled across existing networks, enabling access to a vast amount of sensor data that can be used for condition monitor- ing. However, the traditional way of gaining knowledge and wisdom, by the expert, for designing condition monitoring methods is unfeasible for fully uti- lizing and digesting this enormous amount of information. It does not scale well to complex systems with a huge amount of components and subsys- tems. Therefore, a more automated approach that relies on human experts to a lesser degree, being capable of discovering interesting patterns, generating models for estimating the health status of the equipment, supporting mainte- nance scheduling, and can scale up to many equipment and its subsystems, will provide great benefits for the industry. This thesis demonstrates how to utilize the concept of "Wisdom of the Crowd", i.e. a group of similar individuals, for fault detection and prognosis. The approach is built based on an unsupervised deviation detection method, Consensus Self-Organizing Models (COSMO). The method assumes that the majority of a crowd is healthy; individual deviates from the majority are con- sidered as potentially faulty. The COSMO method encodes sensor data into models, and the distances between individual samples and the crowd are measured in the model space. This information, regarding how different an individual performs compared to its peers, is utilized as an indicator for es- timating the health status of the equipment. The generality of the COSMO method is demonstrated with three condition monitoring case studies: i) fault detection and failure prediction for a commercial fleet of city buses, ii) prog- nosis for a fleet of turbofan engines and iii) finding cracks in metallic mate- rial. In addition, the flexibility of the COSMO method is demonstrated with: i) being capable of incorporating domain knowledge on specializing relevant expert features; ii) able to detect multiple types of faults with a generic data- representation, i.e. Echo State Network; iii) incorporating expert feedback on adapting reference group candidate under an active learning setting. Last but i ii not least, this thesis demonstrated that the remaining useful life of the equip- ment can be estimated from the distance to a crowd of peers. Acknowledgments First and foremost, for their dedication and patience, I would like to express my sincere gratitude to my principal supervisor Prof. Thorsteinn Rögnvalds- son and co-supervisor Dr. Sławomir Nowaczyk for their endless support and providing me this opportunity to explore in research. I have learned a lot from them, and I am grateful for all their guidance. I want to express my gratitude to co-authors of the papers included in this thesis, Dr. Xudong Teng, Dr. Eric Antonelo, Dr. Sepideh Pashami, Ece Calikus, Kunru Chen and Dr. Anita Sant’Anna for the collaboration. It is a pleasure to work with all of you. My special thanks to Dr. Sepideh Pashami and Dr. Mohamed-Rafik Bouguelia for their inspirations, advice on my research, and feedback on my thesis. I would also like to thank my mentor Ervin Omerspahic and Klas Thun- berg, at Volvo Bus Corporation, for their support, advice, and discussion in our collaboration. Many thanks to Dr. Stefan Byttner and Prof. Håkan Pet- tersson for their great support as the director of doctoral education. I also would like to thank my support committee members, Prof. Alexey Vinel, and Fredrik Bode, for providing feedback on my research and progress. Prof. Antanas Verikas, Prof. Josef Bigun, Dr. Martin Cooney, Dr. Fernando Alonso-Fernandez, and Dr. Nicholas Wickström have provided me with in- spirations and advice. I want to acknowledge Dr. Björn Åstrand and Dr. Saeed Gholami Shahbandi for their excellent supervision on my master thesis project. Thank you, Roland Thörner, Tommy Salomonsson, Dr. Eren Erdal Aksoy, Dr. Reza Khoshkangini, Dr. Peyman Mashhadi, Dr. Mahmoud Rahat, and Mo- hammed Ghaith Altarabichi for all the interesting conversations and discus- sions. Many thanks to Eva Nestius, Stefan Gunnarsson, and Jessika Rosen- berg for administrative support. I feel very lucy and privileged to be part of the Center for Applied Intelligent Systems Research (CAISR), the Intelligent Systems and Digital Design Laboratory (ISDD) and Embedded and Intelli- gent Systems Industrial Graduate School (EISIGS), the School of Information Technology (ITE) at Halmstad University. I must express my gratitude to all my colleagues for their assistance and all the informative discussions. iii iv I would like to thank Daniel Reimhult, Elham Pirnia, Evangelia Soultani, Thomas Hordern, and Jens Lundström for their assistance and fruitful dis- cussions at Volvo Bus Corporation and Volvo Group Connected Solutions. Special recognition goes to my family for their unconditional support and love. Words cannot express how grateful I am... Last but not least, my grati- tude to all my friends and fellow labmates, thank you all for being supportive and backing me up! Thank you Maytheewat Aramrattana for all the moments we shared since the time we studied our MSc program. Thank you, Jennifer David and Kevin Hernández Dáaz, for the late-night brainstormings we had in the lab. Thank you Hassan Nemati, Ece Calikus, Pablo del Moral, Awais Ashfaq, Deycy Janeth Sanchez Preciado, Süleyman Savas, Mahsa Varshosaz, Yingfu Zeng, and Siddhartha Khandelwal for all the wonderful times we had. Thank you Iulian Carpatorea for all the interesting conversations and discus- sions we had. My special thanks to Viktor Vasilev for his help, support and the great time we have shared with Yod, Fei Xu, and Carlos Fuentes. Thank you for being supportive and informative Jiajun Qu and Kan Chen. Many thanks to all. ;-) List of Publications This thesis summarizes the following papers: I. Yuantao Fan, Sławomir Nowaczyk, and Thorsteinn Rögnvaldsson. Eval- uation of self-organized approach for predicting compressor faults in a city bus fleet. volume 53, pages 447–456. Elsevier, 2015. II. Yuantao Fan, Sławomir Nowaczyk, and Thorsteinn Rögnvaldsson. In- corporating expert knowledge into a self-organized approach for pre- dicting compressor faults in a city bus fleet. In Thirteenth Scandinavian Conference on Artificial Intelligence: SCAI 2015, volume 278, pages 58–67. IOS Press, 2015. III. Yuantao Fan, Sławomir Nowaczyk, Thorsteinn Rögnvaldsson, and Eric Ais- lan Antonelo. Predicting air compressor failures with echo state net- works. In Third European Conference of the Prognostics and Health Man- agement Society 2016, Bilbao, Spain, 5-8 July, 2016, pages 568–578. PHM Society, 2016. IV. Xudong Teng, Xin Zhang, Yuantao Fan, and Dong Zhang. Evaluation of cracks in metallic material using a self-organized data-driven model of acoustic echo-signal. Applied Sciences, 9(1):95, 2019. V. Yuantao Fan, Sławomir Nowaczyk, and Thorsteinn Rögnvaldsson. Trans- fer learning for remaining useful life prediction based on consensus self- organizing models. Submitted to Reliability Engineering & System Safety, 2019. VI. Ece Calikus, Yuantao Fan, Slawomir Nowaczyk, and Anita Sant’Anna. Interactive-cosmo: Consensus self-organized models for fault detection with expert feedback. In Proceedings of the Workshop on Interactive Data Mining, page 5. ACM, 2019. VII. Kunru Chen, Sepideh Pashami, Yuantao Fan, and Slawomir Nowaczyk. Predicting air compressor failures using long short term memory net- v vi works. In EPIA Conference on Artificial Intelligence, pages 596–609. Springer, 2019. Contents 1 Introduction 1 1.1 Motivations .............................. 2 1.2 Objectives ............................... 2 1.3 Challenges ............................... 3 1.4 Research Gap and Questions .................... 4 1.5 Applying a Wisdom of the Crowd Approach for Fault Detec- tion and Prognosis .......................... 7 1.5.1 Fault Detection and Failure Prediction for a Fleet of City Buses .............................. 8 1.5.2 Prognosis for a Fleet of Turbofan Engines ......... 9 1.5.3 Non-Destructive Testing ................... 10 1.6 Overview of the Contribution .................... 10 2 Background and Related Works 13 2.1 Condition Monitoring and Maintenance Strategies ........ 13 2.1.1 Terminology .......................... 13 2.1.2 Fault Detection and Prognosis ............... 15 2.1.3 Maintenance Strategies ................... 19 2.2 On Applying Machine Learning for Condition Monitoring ... 21 2.3 Fleet based Approaches for Fault Detection and Prognostics .. 24 2.4 Representation Learning and Deviation Detection in Model Space 25 3 Methodology 29 3.1 Data Representations ......................... 29 3.2 The Construction of the Crowd ................... 33 3.3 Measuring Distance between Samples ............... 35 3.4 Compute Deviation Level ...................... 37 3.5 Compute Distance to Peers ..................... 38 3.6 Evaluating