Design of a Distributed System for Information Processing in a Mist-Edge-Cloud Environment Moisés Carral Ortiz
Total Page:16
File Type:pdf, Size:1020Kb
Design of a Distributed System for Information Processing in a Mist-Edge-Cloud Environment Moisés Carral Ortiz Master’s Thesis presented to the Telecommunications Engineering School Master’s Degree in Telecommunications Engineering Supervisors Felipe José Gil Castiñeira Juan José López Escobar 2020 - 2021 Acknowledgements I would like to express my gratitude to my tutor, Felipe Gil, who has allowed me to discover the unknown world of research and has supported me both professionally and personally to make this thesis what it is. Also I would like to consider the good working environment created by my co-workers in GTI researh group. Finally, thanks to my virtuous and tireless master’s degree teammates. Vigo, July 20, 2021 iii Abstract The exponential growth of connected devices as well as massive data generation are challenging well-established technologies such as Cloud Computing. The Big Data paradigm for inferring knowledge and predicting results supports the sensorisation of environments in all areas (social, business, environmental, etc.) to obtain more information for the training of artificial intelligences or machine learning models. The need for low latency and high throughput applications in sensor networks is driving the emergence of new computational paradigms and network architectures. Real-time systems in which decision-making is critical, such as autonomous driving or the management of automated processes by robots in Industry 4.0, cannot adapt their needs to Cloud Computing. Other paradigms such as Mist Computing or Fog Computing are emerging in response to these needs to evolve actual computing architectures. This paper introduces some of the proposals that, in contrast to the classical structure of centralised data processing in the Cloud, advocate for a distributed architecture that brings the IoT-Edge-Cloud Continuum. This master thesis is part of the collaboration of the Grupo de Tecnologías de la Información research group in the European 2020 NextPerception project. The aim of this collaboration is the creation of an intelligent distributed platform. Specifically, this thesis is focused on on the design of a distributed network communication architecture, proposing, verifying and contributing to a novel technology called Zenoh. The reader will understand how the protocol works and will understand the contributions we made towards improving it. Key words: Zenoh, Data-centric, Publish, Subscribe, Fog computing, Mist computing, Distributed Computing v Contents Acknowledgements iii Abstract v List of figures xi 1 Introduction1 1.1 Context.....................................1 1.2 NextPerception.................................2 1.3 Objectives....................................3 2 State of the art5 2.1 Introduction...................................5 2.2 From centralised to distributed architectures.................6 2.2.1 Centralised architectures........................6 2.2.2 Hierarchical architectures.......................7 2.2.3 Distributed architectures........................9 2.3 Data distribution................................ 11 2.4 Embedded real-time operating systems.................... 15 3 Design of a distributed computing architecture 17 3.1 Fusion strategies and computational load distribution............ 18 3.2 Architecture................................... 20 4 Data distribution 27 4.1 Zenoh...................................... 27 4.2 Zenoh protocol................................. 31 4.3 Zenoh with RTOS............................... 34 4.4 Wireshark dissector............................... 35 4.5 Zenoh in NextPerception............................ 38 5 Results and validation 41 5.1 Metrics..................................... 41 5.1.1 Throughput............................... 42 vii Contents 5.1.2 Latency................................. 46 5.2 Zenoh over BLE................................ 47 6 Conclusions 51 6.1 Future work................................... 52 Bibliography 53 Appendices 57 Zenoh demonstrator 59 A.1 Prerequisites.................................. 59 A.1.1 OpenCV................................. 59 A.1.2 Rust nightly............................... 61 A.1.3 Zenoh-python.............................. 61 A.2 Deployment................................... 61 A.2.1 Peer deployment............................ 61 A.2.2 Mixed topology............................. 65 A.3 Zenoh router installation............................ 70 A.3.1 Linux (x86_64)............................. 70 A.3.2 ARM (arm64).............................. 70 Protocol Messages 73 B.1 Zenoh Messages................................. 73 B.2 Session Messages................................ 74 viii Acronyms ABS Antiblockiersystem. AI Artificial Intelligent. AMQP Advanced Message Queuing Protocol. ANSI American National Standards Institute. BLE Bluetooth Low Energy. CDN Content Delivery Network. CoAP Constrained Application Protocol. CPU Central Processing Unit. DSS Data Distribution Service. FAT File Allocation Table. FIM Fog Infrastructure Manager. GATT Generic Attribute Profile. GPIO General Purpose Input/Output. HLC Hybrid Logical Clock. IDEs Integrated Development Environmets. IoT Internet Of Things. IP Internet Protocol. IT Information Technology. KVM Kernel-based Virtual Machine. ix Acronyms LED Light-Emitting Diode. LSB Least Significant Bit. LXC LinuX Containers. MANO Management and Network Orchestration. ML Machine Learning. MPUs Mutiple Process Units. MQTT Message Queuing Telemetry Transport. MSB Most Significant Bit. NDN Named Data Networks. NFV Network Functions Virtualization. OT Operation Technology. POSIX Portable Operating System Interface. QoS Quality of Service. QUIC Quick UDP Internet Connections. RAM Random Access Memory. REST Representational State Transfer. RPC Remote Procedure Call. RTOS Real-Time Operating Systems. TCP Transmission Control Protocol. TPU Tensor Processing Unit. UUID Universally Unique IDentifier. V2V Vehicle to Vehicle. V2X Vehicle to Everything. x List of Figures 1.1 Basic architecture of the UC1 from NextPerception.............4 2.1 Processing location selection..........................7 2.2 Fog Cloud Computing architecture [8].....................8 2.3 Distributed computing architecture......................9 2.4 Event based information delivery model.................... 12 2.5 DDS communication scheme [14]........................ 13 2.6 Protocols comparison table [17]......................... 14 3.1 Complete architecture [27]........................... 22 3.2 Design of NextPerception stack......................... 25 3.3 FogØ5 in the UC1................................ 25 3.4 Zenoh FogØ5 in the UC3............................ 26 4.1 Peer, client and router configurations..................... 28 4.2 Zenoh stack diagram [18]............................ 29 4.3 Zenoh messages interchanged during communication............. 32 4.4 Zenoh over TCP................................. 36 4.5 Zenoh over UDP................................. 36 4.6 Wireshark’s Zenoh dissector without heuristics................ 37 4.7 Wireshark’s Zenoh dissector with heuristics.................. 38 4.8 Diagram of demonstrator presented to the consortium............ 39 4.9 Partial solution to the partner’s problem................... 40 4.10 Final architecture of the integration with Zenoh............... 40 5.1 Zenoh throughput (Millions of messages per second)............. 43 5.2 Zenoh throughput (GB/s)............................ 43 5.3 Comparison between Zenoh P2P and Zenoh brokered............ 44 5.4 Throughput in MB/s.............................. 45 5.5 Throughput in Mmsg/s............................. 45 5.6 Zenoh-net API in P2P mode (localhost).................... 47 5.7 Zenoh API in P2P mode (localhost)...................... 48 5.8 Zenoh-net API in client mode (localhost)................... 48 5.9 Zenoh API in client mode (localhost)..................... 49 xi List of Figures 1 P2P deployment diagram............................ 62 2 Example of a mixed Zenoh topology...................... 65 3 Architecture of the example........................... 66 xii 1 Introduction 1.1 Context Since its existence, the human being has always sought ways to understand and resolve the problems encountered during evolution. The collection and the analysis of the information usually helped to infer and finally predict, to foresee and act even before the problems appear. Different tools have been used to help in this task. The developments in computing made it possible the creation of what is called Artificial Intelligent(AI) and Machine Learning(ML) to infer and predict actions or results. Such technologies require large amounts of information, for example collected by sensors, that have to be processed in large infrastructures. Nowadays, the most popular approach is Cloud Computing [1], which brought advantages such as virtualised resources, dynamic and scalable services, resources on demand, elas- ticity, etc. However, Cloud Computing usually follows a centralised approach, requiring collecting the data from Internet Of Things(IoT) networks through the Internet to a centralised location. Data flowing between private networks to the Cloud (usually pro- vided by a third-party company) could endanger the privacy of our data. Also, the huge amount of data flowing to a Cloud could collapse the actual data networks. Furthermore, the latency introduced by the communication with the Cloud and the processing time may be unacceptable for real-time applications. A recent example of the vulnerability of the Cloud is the failure