Evaluating DDS, MQTT, and Zeromq Under Different Iot Traffic Conditions
Total Page:16
File Type:pdf, Size:1020Kb
Evaluating DDS, MQTT, and ZeroMQ Under Different IoT Traffic Conditions Zhuangwei Kang Robert Canady Abhishek Dubey Vanderbilt University Vanderbilt University Vanderbilt University Nashville, Tennessee Nashville, Tennessee Nashville, Tennessee [email protected] [email protected] [email protected] Aniruddha Gokhale Shashank Shekhar Matous Sedlacek Vanderbilt University Siemens Technology Siemens Technology Nashville, Tennessee Princeton, New Jersey Munich, Germany [email protected] [email protected] [email protected] Abstract Keywords: Publish/Subscribe Middleware, Benchmark- ing, MQTT, DDS, ZeroMQ, Performance Evaluation Publish/Subscribe (pub/sub) semantics are critical for IoT applications due to their loosely coupled nature. Although OMG DDS, MQTT, and ZeroMQ are mature pub/sub solutions used for IoT, prior studies show that their performance varies significantly under different 1 Introduction load conditions and QoS configurations, which makes Distributed deployment of real-time applications and middleware selection and configuration decisions hard. high-speed dissemination of massive data have been hall- Moreover, the load conditions and role of QoS settings in marks of the Internet of Things (IoT) platforms. IoT prior comparison studies are not comprehensive and well- applications typically adopt publish/subscribe (pub/- documented. To address these limitations, we (1) propose sub) middleware for asynchronous and cross-platform a set of performance-related properties for pub/sub mid- communication. OMG Data Distribution Service (DDS), dleware and investigate their support in DDS, MQTT, ZeroMQ, and MQTT are three representative pub/sub and ZeroMQ; (2) perform systematic experiments under technologies that have entirely different architectures (de- three representative, lab-based real-world IoT use cases; centralized data-centric, decentralized message-centric, and (3) improve DDS performance by applying three and centralized message-centric, respectively). All of of our proposed QoS properties. Empirical results show them implement the pub/sub messaging pattern and that DDS has the most thorough QoS support, and more provide a set of configurable parameters for customizing reliable performance in most scenarios. In addition, its middleware behaviors and resource allocation. Accord- Multicast, TurboMode, and AutoThrottle QoS policies ingly, an essential question that needs to be answered can effectively improve DDS performance in terms of is how to choose an appropriate middleware given a throughput and latency. workload condition, and which parameters should be CCS Concepts: • Software and its engineering → Mes- regarded for making the optimal configuration decisions. sage oriented middleware; Publish-subscribe / event- To that end, we propose a set of QoS properties that based architectures; • General and reference → Evalu- are tied to the performance of IoT pub/sub applications ation. and investigate which of those properties are supported by DDS, MQTT, and ZeroMQ. We then conduct a sys- tematic set of experiments to assess their performance Permission to make digital or hard copies of all or part of this in three pub/sub use cases (high-frequency, periodic, work for personal or classroom use is granted without fee provided and sporadic), which provides us with baselines for do- that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation ing further performance optimization. Empirical results on the first page. Copyrights for components of this work owned show that DDS has the most stable performance in the by others than ACM must be honored. Abstracting with credit is above scenarios and provides the most in-line support permitted. To copy otherwise, or republish, to post on servers or to the QoS proprieties we propose. To understand the to redistribute to lists, requires prior specific permission and/or a performance under high frequency data flows, we further fee. Request permissions from [email protected]. M4IoT ’20, Dec 7-11, 2020, Delft, The Netherlands experimentally explore the impact of three competitive © 2020 Association for Computing Machinery. QoS policies (Multicast, TurboMode, and AutoThrottle) ACM ISBN 9781450370288. $15.00 on improving DDS application performance under such conditions. M4IoT ’20, Dec 7-11, 2020, Delft, The Netherlands Kang, et al. 2 QoS Properties Essentials for IoT Systems lack of strict reliability guarantee in ZeroMQ’s pub/sub The primary purpose of this section is to introduce sev- pattern, messages sent in a large batch risk being lost eral performance-related QoS policies available in DDS, when the network is crowded. At the subscriber side, if MQTT, and ZeroMQ that are necessary for performance- subscriber can keep up with publisher and there is no sensitive IoT applications. As a branch of networking application-level queue backlog, ZeroMQ would turn off applications, the performance (throughput and latency) its batching function for the sake of lower end-to-end of pub/sub middleware mainly depends on the network delay. Otherwise, the batching would be enabled to help congestion status and internal resource configuration. the worker thread promote the speed of flushing back- In this paper, we look at the network aspect, and the logged messages. MQTT does not support intelligent principle we followed to choose these features is whether batching. it can effectively change the amount of data entering 2.4 Rate Limit and leaving the network pipeline per unit time or the processing logic of messages in the network. Rate Limit is a flow control mechanism that protects network resources from being encroached by malicious ac- 2.1 Reliability tors in an IoT cluster by specifying the maximum rate at Reliability policy determines whether messages can be re- which a publisher may send samples to the network. DDS liably transmitted, which fatally affect the data-processing allows users to define the Custom Flow Controller that and decision-making processes of IoT applications. DDS maintains a separate FIFO queue for each connection provides two types of transmission guarantees: Best Ef- in which data instances generated by asynchronous pub- fort (i.e., no guarantees) and Reliable (i.e., retry until lishers can be placed. ZeroMQ supports rate limit only success). MQTT provides three service levels: QoS 0, when the multicast protocol (PGM/EPGM) is enabled, QoS 1, QoS 2, representing "at-most-once", "at-least- and is implemented by setting the multicast window size. once", and "exactly-once", respectively. MQTT’s QoS 0 MQTT server shapes the egress traffic of each TCP/SSL is equivalent to DDS’ BestEffort and QoS 2 to DDS’ Re- connection based on the Leaky Bucket (LB) algorithm. liable delivery. ZMQ pub/sub pattern may lose messages when the network is congested. 2.5 Message LifeSpan 2.2 Multicast Message expiration mechanism restrains the longest time that a message is regarded as a legal one in the sys- Multicast allows IoT applications to scale better when tem, which improves overall memory utilization rate multiple subscribers exist. Both DDS and ZeroMQ allow and avoids delivering stale data. In the reliable trans- users to set the number of hops that multicasting mes- port mode of DDS, if the LifeSpan QoS policy is not sages can traverse, which intends to avoid flooding large configured, unconfirmed data instances will reside in networks with multicast traffic. Although MQTT runs the publisher’s writing buffer for a long time and be over unicast protocol, it emulates the application-layer re-transmitted endlessly, which not only wastes mem- "multicast" by wrapping up point-to-point TCP connec- ory and bandwidth, but also destroys the timeliness tions, which is also known as multicast-over-unicast. The of messages. Adhering to similar intent, MQTT imple- protocol utilized in ZeroMQ for multicast include Prag- mentations(etc. EMQ X, HiveMQ, and VerneMQ) that matic General Multicast (PGM) [5] and Encapsulated adopt the latest MQTT specification (MQTT 5.0) al- Pragmatic General Multicast (EPGM). lows publishers to specify a property for each sample 2.3 Intelligent Batching called message-expiry-interval that defines the actual Message batching intends to improve the throughput of lifespan of messages in seconds. There is no analogies IoT applications. It avoids frequent system calls through with LifeSpan QoS in current ZeroMQ implementations. the network stack due to message by message process- 2.6 Topic Priority ing and can be performed at a fixed interval or over number of messages. The TurboMode QoS in DDS au- Topic priority aims to distinguish the importance of tomatically decides the optimal number of bytes in a different messages, which ensures that the overall utility batch on publisher side based on the data sample size, of a multi-topic IoT system is maximized under given writing speed and the real-time system state. Message resource limitation. DDS enables this property via the batching in ZeroMQ is can be enabled at either pub- Tansport_Priority policy that is signified by a 32-bit lisher or subscriber side.Rather than deciding timeout or signed integer. An analogous parameter in ZeroMQ is batch size for a single chunk, ZeroMQ always forwards called ZMQ_TOS that is implemented in the same logic all messages queued in memory at the moment to the as DDS. Unlike the above system-level implementations, network interface card in one go. However,