RocketBufs: A Framework for Building Efficient, In-Memory, Message-Oriented Middleware Huy Hoang, Benjamin Cassell, Tim Brecht, Samer Al-Kiswany Cheriton School of Computer Science, University of Waterloo ABSTRACT ACM Reference Format: As companies increasingly deploy message-oriented middleware Huy Hoang, Benjamin Cassell, Tim Brecht, Samer Al-Kiswany. 2020. Rock- (MOM) systems in mission-critical components of their infrastruc- etBufs: A Framework for Building Efficient, In-Memory, Message-Oriented Middleware. In The 14th ACM International Conference on Distributed and tures and services, the demand for improved performance and func- Event-based Systems (DEBS ’20), July 13–17, 2020, Virtual Event, QC, Canada. tionality has accelerated the rate at which new systems are being ACM, New York, NY, USA, 12 pages. https://doi.org/10.1145/3401025.3401744 developed. Unfortunately, existing MOM systems are not designed to take advantages of techniques for high-performance data center communication (e.g., RDMA). In this paper, we describe the design 1 INTRODUCTION and implementation of RocketBufs, a framework which provides Message-Oriented Middleware (MOM) systems, also often referred infrastructure for building high-performance, in-memory Message- to as publish/subscribe, message-queuing, or event-based systems, Oriented Middleware (MOM) applications. RocketBufs provides are a popular class of software designed to support loosely-coupled memory-based buffer abstractions and APIs, which are designed messaging in modern distributed applications. Examples of appli- to work efficiently with different transport protocols. Applications cations and services that utilize MOM systems include IBM’s cloud implemented using RocketBufs manage buffer data using input functions [29], the Apache OpenWhisk serverless framework [10], (rIn) and output (rOut) classes, while the framework is responsible the Hyperledger blockchain framework [7] and streaming media for transmitting, receiving and synchronizing buffer access. systems [60]. These applications all follow a produce-disseminate- We use our current implementation, that supports both TCP and consume (PDC) design pattern, where one or more producers (or RDMA, to demonstrate the utility and evaluate the performance of publishers) send data as messages to an MOM substrate (often com- RocketBufs by using it to implement a publish/subscribe message prised of message brokers) for dissemination to a possibly large queuing system called RBMQ and a live streaming video application. number of consumers (or subscribers). To scale to high loads some When comparing RBMQ against two widely-used, industry-grade MOM systems support the distribution of messages to other brokers MOM systems, namely RabbitMQ and Redis, our evaluations show using topologies best suited to the application. that when using TCP, RBMQ achieves broker messaging through- Many modern applications have high performance and scalabil- put up to 1.9 times higher than RabbitMQ and roughly on par with ity demands with regard to message delivery. Facebook’s pub/sub that of Redis, when configured comparably. However, RBMQ sub- system, for example, delivers over 35 Gigabytes per second within scribers require significantly less CPU resources than those using their event processing pipeline [63], while Twitch, a live streaming Redis, allowing those resources to be used for other purposes like video service, handles hundreds of billions of minutes worth of processing application data. When configured to use RDMA, RBMQ video content a year [22]. Other classes of applications, such as provides throughput up to 3.7 times higher than RabbitMQ and up online gaming [23] and stock trading [67], require messages to to 1.7 times higher than Redis. We also demonstrate the flexibility be delivered with extremely low latency. As a result, constructing of RocketBufs by implementing a live streaming video service and high-performance and scalable MOM systems to deal with emerg- show that it can increase the number of simultaneous viewers by ing workloads is of great interest in both industry and academia. up to 55%. Over the past five years alone, many new open-source MOM sys- tems have been developed [9, 11, 20, 38, 61, 68], and cloud providers CCS CONCEPTS have continued to introduce new MOM services as part of their • Networks ! Programming interfaces; • Computing method- infrastructure [3, 4, 24, 25]. ologies ! Distributed computing methodologies; • Information Often in MOM deployments, data is moved between hosts that systems ! Data management systems. reside within a data center [39, 62]. One approach to building high-performance MOM systems for such environments is to lever- KEYWORDS age data center networking capabilities, especially those that of- message-oriented middleware, in-memory, event-based systems, fer kernel-bypass features to enable high-throughput and/or low- message queuing, publish subscribe, live streaming video latency communication such as Remote Direct Memory Access (RDMA) [44] or emerging technologies, accelerators and APIs such as DPDK [69], F-Stack [21], and Solarflare/TCPDirect [64]. Such DEBS ’20, July 13–17, 2020, Virtual Event, QC, Canada technologies are becoming increasingly important to reduce mes- © 2020 Association for Computing Machinery. This is the author’s version of the work. It is posted here for your personal use. Not sage dissemination costs to the rapidly growing numbers of brokers for redistribution. The definitive Version of Record was published in The 14th ACM and subscribers required by modern applications. Unfortunately, International Conference on Distributed and Event-based Systems (DEBS ’20), July 13–17, 2020, Virtual Event, QC, Canada, https://doi.org/10.1145/3401025.3401744. commonly-used MOM systems do not take advantage of these ca- pabilities and instead use kernel-based TCP, which in many cases DEBS ’20, July 13–17, 2020, Virtual Event, QC, Canada Huy Hoang, Benjamin Cassell, Tim Brecht, Samer Al-Kiswany incurs protocol processing and copying overhead (even for com- we believe that other transport layer APIs and technologies like munication within a data center [27]), limiting throughput and QUIC [31], DPDK [69], F-Stack [21], and Solarflare/TCPDirect [64] resulting in higher latency. could also be efficiently supported by the framework, providing One reason current MOM systems avoid using emerging tech- benefits to all applications built using RocketBufs. This paper makes nologies is the complexity of supporting multiple communication the following contributions: substrates in a single MOM implementation. For instance, the native • We describe RocketBufs, a framework that facilitates the con- RDMA verb interface uses abstractions and APIs fundamentally struction of scalable, high-performance Message-Oriented different from the socket abstraction that are quite complicated to Middleware systems. use [1]. As a result, significant engineering effort would be required • We describe a prototype implementation of RBMQ, a pub- for both new and existing MOM systems to support RDMA and lish/subscribe messaging system built on top of RocketBufs. TCP, since two separate data transfer implementations would be re- We evaluate the performance of RBMQ by comparing it quired. Similarly, emerging technologies often initially provide new against RabbitMQ and Redis, two widely-used, industry- abstractions and custom APIs that require additional modifications grade MOM systems. to the MOM system implementation (e.g., DPDK [69]). • We demonstrate the flexibility of RocketBufs by using it to We believe it is important for MOM systems to utilize RDMA build a second, quite different, application to support the and other existing and emerging technologies designed for high- replication and delivery of live streaming video. Our empiri- throughput and/or low-latency data center communication while cal evaluation shows that RocketBufs is able to support up to not being forced to implement code specific to each technology. To 27% and 55% higher simultaneous viewer throughput than this end, we propose RocketBufs, a framework to facilitate the easy Redis when using TCP and RDMA, respectively. construction of high-performance in-memory Message-Oriented Middleware systems. RocketBufs provides a natural memory-based 2 RELATED WORK buffer abstraction. Application developers control the transmis- sion and reception of data using input (rIn) and output (rOut) Networking Technologies and Libraries: The desire to provide classes that are associated with buffers. RocketBufs implements efficient access to networking resources is long-standing concern and handles the transfers of buffered data between communicating and several technologies and libraries have been created for this nodes and provides mechanisms for flow control. This design al- reason. In order to provide the best performance possible these lows flexible topologies to be constructed for scaling MOM systems. technologies are often accompanied with new abstractions and RocketBufs’ APIs are agnostic to the underlying transport layer and APIs. In order to provide some benefits to legacy applications new developers can configure the framework to use different protocols libraries are often created on top of the “native” APIs that attempt or technologies without changing application code. to mimic socket abstractions and APIs. Remote Direct Memory Figure 1 shows that applications can be built using RocketBufs Access (RDMA) is one such example with the native verbs interface which