Analytical Assessment of Binary Data Serialization Techniques in IoT Context [Evaluating Protocol Buffers, FlatBuffers, MessagePack, and BSON for Sensor Nodes] Amrit Kumar Biswal Obada Almallah Student Id: 873052 Student Id: 894304 Advisor: Prof. Marco Brambilla Dipartimento di Elettronica, Informazione e Bioingegneria Politecnico di Milano This thesis is submitted for the degree of Master of Science in Computer Science and Engineering December 2019 We would like to dedicate this thesis to our families, near and far. Acknowledgements First and foremost, we would like to express our appreciation to the power of teamwork and synchronized efforts that led the execution of this thesis. We would like to express our gratitude to our supervisor Prof. Marco Brambilla, who guided us in the right direction and helped us overcome the obstacles faced. Moreover, we would like to express our appreciation and acknowledge the power and vigorous efforts of the open-source community in developing and maintaining technologies that helped us conduct our experiments. Last but not least, we would like to thank our families and friends for the support, motivation, and inspiration they empowered us with that helped us stay on track. Special thanks to our friend Giacomo Tento for providing us with the Italian translated abstract. Abstract The exceptional development of IoT-enabled technologies has been pervasively fostering smart solutions that drive the innovation in various industries. This wide-scale and rapid development in the IoT market faces different challenges related to interoperability, due to fragmented technologies and applications, and related to the need of more IoT-specific standardization applied to the IoT infrastructure that considers the constraints and limitations of IoT applications. These issues can be traced back to the need of achieving a better resource efficiency in IoT systems due to the resource-constrained nature of IoT devices andthe limitations posed by the low power networks bandwidth, which in turn induces the IoT industry to apply different measures in order to achieve efficiency on the devices and on the network. These measures aim at reducing the overall traffic at the network level as well as reducing power and memory consumption at the device level. By achieving optimal network and device settings, the IoT system can in turn have the capacity to offer more services on the application level which consequently accelerates the achievement of the vision and expectations of IoT. One of the operations affecting device-level efficiency lies in the mechanism applied to structure and serialize the payload data. In this thesis we evaluate the resource-efficiency of a group of binary-based mechanisms in dealing with sensor nodesdata in purpose of arriving to candidate technique(s) that have the potential of being standardized within the IoT infrastructure as interoperable data format protocol(s). Keywords: IoT, Internet-of-things, Performance, Evaluation, Binary, Semantic, Data Serialization, Protocol Buffers, FlatBuffers, MessagePack, BSON Sommario L’eccezionale sviluppo di tecnologie abilitate all’IoT ha pervasivamente promosso l’adozione di smart solutions che guidano l’innovazione in vari settori industriali. Questo rapido e consistente sviluppo avvenuto del mercato IoT deve tuttavia affrontare, a causa della fram- mentazione di tecnologie e applicazioni utilizzate, diverse sfide legate all’interoperatibilita’, collegate alla necessita’ di una standardizzazione maggiormente specifica applicata all’infra- struttura IoT che tenga conto di vincoli e limitazioni delle applicazioni IoT. Tali questioni hanno radici nella necessita’ di un uso piu’ efficiente delle risorse disponibili nei sistemi IoT, a causa della limitatezza dei mezzi dovuta alla natura dei dispositivi IoT e ai vincoli imposti dalla capacita’ delle reti, le quali, d’altra parte, stimolano l’industria IoT ad appliccare differenti soluzioni volte all’efficienza di tali dispositivi su tali reti. Queste soluzioni mirano alla riduzione del traffico complessivo a livello della rete e al contempo a limitare l’utilizzo energetico e di memoria a livello del dispositivo. Con l’utilizzo di impostazioni ottimali sia per la rete che per il dispositivo il sistema IoT diventa in grado di fornire maggiori servizi a livello di applicazioni, i quali di conseguenza accelerano l’avvicinamento verso l’obiettivo dell’IoT. Una delle operazioni che influenzano l’efficienza a livello di dispositivo si trova nel meccanismo utilizzato nella strutturazione e serializzazione del carico di dati. In questa tesi, valutiamo un gruppo di meccanismi a base binaria sulla base della loro efficienza nel confrontarsi con dati dei sensor nodes con l’obiettivo di isolare le tecniche con il potenziale di essere standardizzate all’interno dell’infrastruttura IoT come protocolli interoperativi di data format. Contents Acknowledgments iv Abstract v Sommario vi 1 Introduction 1 1.1 Context & Motivation . 1 1.2 Research Goals . 3 1.3 Thesis Structure . 4 2 Background 5 2.1 Internet of Things . 5 2.2 Key Features and Challenges . 6 2.3 The Internet of Things Architecture . 8 2.4 Connectivity Protocols/Standards . 9 2.5 Models . 13 2.6 Data Exchange Protocols . 16 2.7 Data Format Protocols . 18 2.7.1 Text-Based Protocols . 19 2.7.2 Binary-based Protocols . 21 3 Related Work 24 3.1 Data Exchange Protocols Evaluation . 24 3.2 Payload Optimization Approaches . 26 3.3 Comparative Discussion . 29 4 Binary Data Serialization in Sensor Nodes 31 4.1 Introduction and Case Statement . 31 4.2 Implemented Protocols . 32 vii 4.3 Performance Metrics . 34 4.3.1 Qualitative Analysis . 34 4.3.2 Quantitative Analysis . 35 5 Hardware Setup and Implementations 37 5.1 NodeMCU Specifications . 38 5.2 Arduino IDE . 39 5.3 Platform Configuration . 39 5.4 Protocol Buffers Implementation . 39 5.5 FlatBuffers Implementation . 40 5.6 MessagePack Implementation . 41 5.7 BSON Implementation . 42 6 Simulation Setup and Implementation 43 6.1 Contiki OS . 43 6.2 Cooja Simulator . 44 6.3 Simulation setup . 45 6.4 Benchmarking . 47 6.5 Nanopb (Protocol Buffers) . 50 6.6 FlatCC (FlatBuffers) . 50 6.7 CWPack (MessagePack) . 51 6.8 BINSON (BSON partial implementation) . 51 7 Experiments and Discussion 52 7.1 Payload Structure Types . 52 7.2 Hardware Runs . 54 7.3 Simulation Runs . 54 7.4 Results . 55 7.4.1 Hardware Results & Discussion . 55 7.4.2 Simulation Results & Discussion . 57 7.5 Qualitative Outlook . 61 8 Conclusions 63 8.1 Workflow Summary . 63 8.2 Critical Discussion . 64 8.3 Future Work . 66 viii List of Tables 7.1 Resulted Payload Size (H/W) . 55 7.2 Serialization Speed (H/W) . 55 7.3 Deserialization Speed (H/W) . 55 7.4 Total Speed Performance (H/W) . 56 7.5 Library Overhead (H/W) . 56 7.6 Total Payload Size (simulation) . 57 7.7 Serialization time (simulation) . 58 7.8 Deserialization time (simulation) . 58 7.9 Library Overhead (simulation) . 58 7.10 Memory Footprint - serialization (simulation) . 58 7.11 Memory Footprint - deserialization (simulation) . 59 7.12 Memory Footprint { Serialization with Deserialization (simulation) . 59 7.13 Average Power Consumption (simulation) . 60 7.14 Qualitative View . 62 ix List of Figures 2.1 IoT Verticals . 6 2.2 IoT Building Blocks . 8 2.3 IoT Application Model . 9 2.4 Bus-Based Model . 14 2.5 Broker-Based Model . 15 6.1 Size command outputs . 48 7.1 Visualizing Average Power Consumption . 61 x Chapter 1 Introduction 1.1 Context & Motivation Internet of Things has been the driving force behind the intelligence embedded in various environ- ments. These complex networks that connect millions of devices and people around the world is the backbone of \Industry 4.0" enabling fast and autonomous bi-directional data communication between machineries and devices. With its constant growth and incorporation into numerous and diverse industries, the expectations and figures of IoT applications and technologies show how it will become an essential infrastructure powering and driving people's daily activities and interactions due to the rapidly increasing number of connected \things". The capabilities and advancements of self-identification, localization, and data acquisition through positioning and sensing technologies remain at the heart of IoT-powered innovations. Substantially, sensor data constitutes the majority of the IoT business logic, since IoT appli- cations rely mainly on sensor data to effectively perform operations of monitoring, control, opti- mization that eventually enable automation capabilities paving the way of novel applications. In an IoT system, these sensor devices can be deployed via Wireless Sensor Networks (WSN) where sensors communicate with each other either in a multi-hop or in a single-hop fashion relying on radio frequency mechanisms to exchange information and then send target data over the internet, or be deployed in a client/server architecture where each sensor node sends data to the application server over the internet that is then processed to offer various services to end-users such as smart parking, smart factory, and smart city applications. Regardless of the IoT system architecture, applications feed intensively on sensors' data and these data harvesting devices have resource constraints related to computing power, memory, energy consumption, and limited bandwidth tolerance. These constraints are due to the nature of sensor nodes that run on batteries and on low powered microprocessors and microcontrollers with limited memory and clock frequency. Also, for IoT environment setting, deployment, and operations to 1 CHAPTER 1. INTRODUCTION 2 achieve the IoT vision and expectations, connected devices need to seamlessly communicate through low-bandwidth networks (e.g. 6LoWPAN) where the data link layer payload is extremely limited (e.g. 81 bytes in IEEE 802.15.4 link.