System of Systems Interoperability Machine Learning Model
Total Page:16
File Type:pdf, Size:1020Kb
LICENTIATE T H E SIS Department of Computer Science and Electrical Engineering Division of EISLAB ISSN 1402-1757 System of Systems Interoperability ISBN 978-91-7790-458-8 (print) ISBN 978-91-7790-459-5 (pdf) Machine Learning Nilsson System of Systems Interoperability Model Jacob Machine Learning Model Luleå University of Technology 2019 Jacob Nilsson Industrial Electronics System of Systems Interoperability Machine Learning Model Jacob Nilsson Dept. of Computer Science and Electrical Engineering Lule˚aUniversity of Technology Lule˚a,Sweden Supervisors: Fredrik Sandin Jerker Delsing Printed by Luleå University of Technology, Graphic Production 2019 ISSN 1402-1757 ISBN 978-91-7790-458-8 (print) ISBN 978-91-7790-459-5 (pdf) Luleå 2019 www.ltu.se ii To Bertil. iii iv Abstract Increasingly flexible and efficient industrial processes and automation systems are de- veloped by integrating computational systems and physical processes, thereby forming large heterogeneous systems of cyber-physical systems. Such systems depend on partic- ular data models and payload formats for communication, and making different entities interoperable is a challenging problem that drives the engineering costs and time to deployment. Interoperability is typically established and maintained manually using do- main knowledge and tools for processing and visualization of symbolic metadata, which limits the scalability of the present approach. The vision of next generation automation frameworks, like the Arrowhead Framework, is to provide autonomous interoperability solutions. In this thesis the problem to automatically establish interoperability between cyber-physical systems is reviewed and formulated as a mathematical optimisation prob- lem, where symbolic metadata and message payloads are combined with machine learning methods to enable message translation and improve system of systems utility. An au- toencoder based implementation of the model is investigated and simulation results for a heating and ventilation system are presented, where messages are partially translated correctly by semantic interpolation and generalisation of the latent representations. A maximum translation accuracy of 49% is obtained using this unsupervised learning ap- proach. Further work is required to improve the translation accuracy, in particular by further exploiting metadata in the model architecture and autoencoder training protocol, and by considering more advanced regularization methods and utility optimization. v vi Contents Part I 1 Chapter 1 { Introduction 3 1.1 Problem Formulation . .5 1.2 Research Methodology . .6 1.3 Thesis Outline . .7 Chapter 2 { Background 9 2.1 Concepts in Future Industrial Automation . .9 2.2 Interoperability in Systems of Systems . 12 2.3 Interoperability and Machine Learning . 15 Chapter 3 { Contributions 19 3.1 Paper A . 19 3.2 Paper B . 20 3.3 Paper C . 20 Chapter 4 { Discussion 23 4.1 Discussion . 23 4.2 Contributions . 23 4.3 Future Work . 25 References 27 Part II 33 Paper A 35 1 Introduction . 37 2 Background . 38 3 Recent Interoperability Research Directions . 40 4 Analysis of Current Research . 43 5 Discussion and Conclusions . 45 6 Acknowledgements . 46 Paper B 51 1 Introduction . 53 2 Interoperability Model . 54 3 Related Work on Interoperability Solutions . 58 vii 4 Mapping to Natural Language Processing . 59 5 Mapping to Graph Neural Networks . 60 6 Discussion: Translator Learning Strategies . 60 7 Concluding Remarks . 62 8 Acknowledgements . 63 Paper C 67 1 Introduction . 69 2 Methods . 72 3 Results . 78 4 Discussion . 82 viii Acknowledgments This thesis contains two years of effort by me and my supervisors, Fredrik Sandin and Jerker Delsing. First, I would like to thank my main supervisor, Fredrik Sandin, for letting me go about this work in my way, and for the good discussions and feedback. Your guidance has most definitely allowed me to evolve as a researcher and I truly feel blessed to have you as my supervisor. Secondly, I would like to thank Jerker Delsing, my co-supervisor and project lead, for allowing me and Fredrik to go and chase ideas that at times seemed so hard to grasp that I wasn't sure what to do. This project has been fun, stressful, agonizing, and awarding. I also want to thank all of my co-workers here at SRT, and in the Productive 4.0 project. You all have made working hours fun and I'm glad we have each other to relieve stress, I am sure most of us would have exploded if not for each other. Next, I would like to thank my mom, dad, sister, and the rest of my family for their support not only during these two years but also the five previous during my master's degree. My maternal grandfather deserves a special mention for always being interested in what I was doing, even if you've had trouble understanding it. Thank you all for being supportive, I know not everybody is as lucky as I am. Now, staying at the same university for seven years means I have a lot of friends who also deserve gratitude. First and foremost, Peder and Adam, I truly feel lucky to have you as friends (and for feeding me on Sundays). Same with Tekniklaget, the five years we spent fixing lights, doing shows, and having fun together were some of the hardest and best times of my life and I am happy for having spent it with you. The board and ledningsgrupp of the Student Union of Engineering and 2017-2018, I had a great time working with you and we kicked ass, no matter what anyone else says. And to everybody else I've befriended over the years: I want to sincerely say thank you for making this time so enjoyable. This thesis represents only half of my journey, I hope to have the continued support of all my friends, family, and coworkers to create great things together! Lule˚a,October 2019 Jacob Nilsson ix x Part I 1 2 Chapter 1 Introduction Digitalization is becoming commonplace in the lives of most. The digital presence shows itself in the increasing number of smart devices around us, as part of a Internet of Things (IoT) [1]. Home assistants (like Amazon Alexa) allows easy access to infor- mation produced by interconnected, smart devices, creating a network of information simplifying home automation. A natural question is \why stop at home automation?", cannot such technologies be used in industrial environments to enable automation to improve condition monitoring, increase production flexibility and cut costs? This idea started the Industry4.0 movement [2], which seeks to enable large-scale interconnected networks of industrial components, from sensors on the factory floor to business manage- ment software. If all parts of a manufacturing process are monitored and connected, the use of modern big-data analysis can be used to cut costs and reduce development times, increasing revenue. The rapid progress of IoT is one of the driving forces behind Industry4.0, but without coordination and supervision IoT devices will not enhance industrial processes. The use of the Industrial Internet of Things (IIoT) in Industry4.0 is enabled by the development of system of systems (SoS) technologies and engineering [3], where the emphasis is put on the interaction between autonomous and highly distributed systems instead of sub- system integrated into monolithic computer systems. The Refence Architectural Model Industrie 4.0 (RAMI4.0) is one effort to design an architecture that can take modern re- quirements of life-cycle management, business layers, and hierarchy levels (see Figure 1.1 into account in a distributed fashion [4], in contrast to the ISA95 hierarchy. A commonly used approach for SoS engineering are service-oriented architectures (SOA) [5], where system functionality is subdivided into units called services which can be consumed by other services to utilize that specific functionality. SOA is implementation-independent, as in any service can be implemented using any language or technology, as long as the described service interface is correct. A central theme in SoS design and engineering is that the component systems are allowed to be heterogeneous in terms of the communication technologies and standards used. For example, the same SoS could contain a temperature sensor using CoAP over 3 4 Introduction Figure 1.1: Illustration of RAMI4.0. WiFi sending data to a database using HTTP over Ethernet. The problem of allowing communication to take place despite varying assumptions about the data model, format, and interfaces used is known as the interoperability problem, and is present in almost every aspect in SoS design and engineering [6]. In this thesis the central problem is the information interoperability problem, which concerns the ability of systems to exchange information [7]. Efficient use of SoS principles has been shown to reduce interoperability problems and therefore cutting engineering costs by as much as 80% for small- to middle scale businesses [8]. Despite the current cost reduction provided by SoS and SOA, such heterogeneous systems are subjected to various interoperability issues, which need to be addressed for further adoption of SoS technologies. Due to increased abilities to collect, store, and process large amounts of data [9], the field of machine learning (and deep learning in particular) has seen a growth in popu- larity and utility. Despite being based on research from the 1990s, it was only in the early 2010s that computer hardware became fast enough to allow training of models with high parameter counts on large datasets [10] while outperforming the previous state-of- the-art (SotA) techniques. Industry 4.0 works in synergy with modern machine learning techniques; vast amounts of data are expected to be produced in Industry 4.0 which can be analyzed with machine learning methods, for example image recognition for manufac- turing lines. Using machine learning solutions presents an interesting approach to solve interoperability problems. However, such solutions have only recently gained some at- tention, and it is this knowledge gap and opportunity to develop more efficient solutions that this thesis focuses on.